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Abstract 

This  thesis  examines  the  impact  of  the  Federal  Acquisition  Streamlining  Act 
(FAS  A)  of  1994  and  the  Clinger-Cohen  Act  on  cost  overruns  in  Department  of  Defense 
(DoD)  contracts.  Many  officials  believe  that  we  must  change  the  way  we  do  business  to 
meet  the  new  post-Cold  War  national  security  challenges.  Changing  the  way  we  do 
business  means  reforming  the  acquisition  process  to  deliver  weapons  systems  faster  and 
cheaper.  The  FASA  and  the  Clinger-Cohen  Act  made  more  changes  to  the  acquisition 
process  than  any  other  policy  had  in  the  ten  years  preceding. 

This  research  effort  studied  220  contracts  completed  between  December  31,  1993 
and  December  31,  2001  to  detennine  if  cost  overruns  on  contracts  completed  before  the 
implementation  of  the  FASA  and  the  Clinger-Cohen  Act  were  different  than  cost 
overruns  on  contracts  completed  after  the  implementation  of  the  FASA  and  the  Clinger- 
Cohen  Act.  The  contracts  were  also  subdivided  to  determine  if  the  results  were  sensitive 
to  acquisition  lifecycle  phase,  branch  of  service,  or  contract  type. 

The  results  indicate  that  cost  overruns  decreased  on  completed  contracts  after  the 
implementation  of  the  legislation.  The  results  were  sensitive  to  the  branch  of  service 
responsible.  Air  Force  contracts  experienced  no  change  in  cost  overruns  after  the 
implementation  of  the  FASA  and  the  Clinger-Cohen  Act,  while  cost  overruns  in  Anny 
and  Navy  contracts  decreased.  The  results  were  not  sensitive  to  lifecycle  phase  or 


contract  type. 
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THE  CLINGER-COHEN  ACT  AND  THEIR  EFFECT  ON  COST  OVERRUNS  IN 
DEPARTMENT  OF  DEFENSE  CONTRACTS 


I.  Introduction 


General  Issue 

September  11,  2001  is  a  day  that  will  not  soon  be  forgotten.  In  a  single  hostile  act 
against  the  United  States,  terrorists  changed  the  world  as  we  know  it.  In  1994,  Colleen 
Preston  made  a  statement  that  the  post-Cold  War  era  introduced  new  political,  economic, 
and  military  security  challenges  to  the  United  States.  In  order  to  meet  these  challenges,  a 
change  in  the  way  we  do  business  is  necessary  (Preston,  1994:7).  That  statement  has 
even  more  impact  now  than  it  did  in  1994. 

Since  we  know  that  change  must  occur,  the  obvious  question  is  what  must 
change?  The  2002  budget  for  government  spending  is  more  than  $2  trillion,  of  which 
$336  billion  is  allocated  for  defense  (OMB,  2003:  Table  S-2).  This  is  more  than  16%  of 
the  overall  budget.  In  order  to  ensure  that  this  money  is  spent  in  the  most  effective 
manner,  certain  controls  are  in  place.  These  controls  influence  the  current  acquisition 
process,  which  protects  taxpayers  from  fraud,  waste,  and  abuse.  Since  the  acquisition 
process  is  central  to  how  the  government  does  business,  one  is  led  to  believe  that 
changing  the  way  the  government  does  business  means  changing  the  acquisition  process. 


General  Lester  Lyles,  Commander  of  Air  Force  Materiel  Command  (AFMC)  commented 

on  change  and  the  acquisition  process. 

Organizations  and  people  either  adapt  to  changing  environments  and  move 
forward,  or  they  become  ineffective  ...  We  must  make  the  cultural 
changes  required  to  be  successful.  We  must  continue  to  deliver 
tomorrow’s  technologies  in  today’s  weapon  systems,  faster,  cheaper,  and 
better  (Lyles,  2002). 

This  research  looks  at  how  we  have  changed  the  acquisition  process  in  order  “to 
deliver  tomorrow’s  technologies  .  .  .  cheaper.”  With  the  Department  of  Defense  (DoD) 
activities  using  such  a  large  share  of  the  federal  budget,  it  is  easy  to  understand  why  cost 
performance  is  a  major  concern  to  those  who  are  interested  in  reforming  the  acquisition 
process.  Delivering  weapon  systems  to  the  Warfighter  in  an  affordable  manner  is  no  easy 
task,  but  it  is  something  we  must  do  in  order  to  meet  the  challenges  of  the  future. 

Specific  Issue 

On  October  13,  1994,  President  Clinton  signed  into  law  the  Federal  Acquisition 
Streamlining  Act  (FASA),  which  was  a  result  of  Vice  President  Gore’s  National 
Perfonnance  Review.  Another  piece  of  legislation,  the  Clinger-Cohen  Act  of  1996,  was 
the  result  of  two  additional  acts,  the  Federal  Acquisition  Refonn  Act  and  the  Information 
Technology  Management  Reform  Act  (DSMC  web  site).  These  two  acts  are  among  the 
myriads  of  legislative  actions  aimed  at  improving  the  way  the  federal  government  does 
business. 

A  major  concern  in  the  area  of  cost  perfonnance  is  the  ability  to  stay  within 
budget,  avoiding  cost  overruns.  Cost  overruns  are  defined  as  the  difference  between  a 
contract’s  actual  costs  at  completion  and  the  budgeted  costs.  The  effect  the  FASA  and 
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the  Clinger-Cohen  Act  have  had  on  controlling  cost  overruns  among  defense  contracts  is 
still  unclear.  The  focus  of  this  research  is  to  quantify  this  effect  and  determine  if  the  cost 
overrun  percentage  has  increased,  decreased,  or  remained  the  same. 

Scope  and  Limitations  of  Study 

The  Defense  Acquisition  Executive  Summary  (DAES)  database  is  a  collection  of 
earned  value  data  for  individual  contracts  on  defense  acquisition  programs  from  1970  to 
2002.  Since  it  captures  all  the  data  required  to  measure  the  fiscal  health  of  a  defense 
acquisition  program,  it  will  be  used  in  this  research  for  the  purpose  of  analyzing  cost 
overruns.  Past  research  has  looked  at  other  acquisition  reform  initiatives  and  their 
subsequent  effect  on  cost  overruns.  These  research  efforts  will  be  summarized  in  Chapter 
2.  In  order  to  center  this  effort  on  the  acquisition  initiatives  in  question,  this  research  will 
be  limited  to  those  contracts  that  were  completed  between  January  1,  1994  and  December 
31,  2001.  Contracts  completed  before  the  pivotal  date  of  December  31,  1997  will  be 
compared  to  contracts  completed  after  the  pivotal  date. 

Additionally,  this  study  focuses  on  those  defense  programs  engaged  in  the  System 
Development  and  Demonstration  phase  and  the  Production  and  Deployment  phase  of  the 
acquisition  life  cycle  (See  Figure  1  below).  Programs  in  the  development  phase  will  be 
compared  to  programs  in  the  production  phase  to  determine  if  there  is  a  significant 
statistical  difference  in  the  respective  cost  overruns  resulting  from  the  enacted  acquisition 
reform  legislation.  Programs  in  other  phases  of  the  acquisition  life  cycle  will  not  be 
addressed. 
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Figure  1.  DoD  5000  Model  (OSD  Website) 

Finally,  this  research  addresses  only  those  programs  that  experienced  cost 

overruns  not  cost  growth.  Cost  growth  occurs  when  a  program  experiences  valid  changes 
that  increase  costs  over  the  course  of  the  procurement  process.  It  is  assumed  for  the 
purpose  of  this  study  that  all  changes  to  the  baseline  are  valid.  Thus,  the  final  budget  will 
be  taken  at  face  value  for  the  purpose  of  analyzing  cost  overruns. 

Research  Questions  and  Methodology 

This  research  benefits  the  DoD  by  providing  senior  leadership  with  analytical, 
quantitative  insight  into  the  cost  overrun  problem  and  how  it  has  changed  as  a  result  of 
specific  legislation  enacted  to  reform  the  acquisition  process.  For  the  full  benefit  to  be 
realized,  it  is  important  that  key  research  questions  are  established.  The  research 
questions  addressed  by  this  study  are  as  follows: 

1 .  Is  the  final  cost  overrun  percentage  on  contracts  completed  before  the 
implementation  of  the  FASA  and  Clinger-Cohen  Act  equal  to  the  final  cost 
overrun  percentage  on  contracts  completed  after  the  implementation  of  the  FASA 
and  Clinger-Cohen  Act? 

2.  Is  the  final  cost  overrun  percentage  sensitive  to  the  phase  of  the  acquisition  life 
cycle  the  program  is  in? 

3.  Is  the  final  cost  overrun  percentage  sensitive  to  the  branch  of  service  responsible 
for  the  program? 
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4.  Is  the  final  cost  overrun  percentage  sensitive  to  the  type  of  contract? 

To  make  a  conclusion  about  whether  or  not  the  DoD  is  performing  better,  worse, 
or  the  same  in  the  area  of  cost  overruns,  hypothesis  testing  procedures  are  used  to 
determine  if  the  differences  are  statistically  significant.  Specifically,  the  research 
hypotheses  to  be  tested  are 

H  i :  The  mean  final  cost  overrun  percentage  on  contracts  completed  before  the 
implementation  of  the  FASA  and  Clinger-Cohen  Act  is  not  equal  to  the  mean 
final  cost  overrun  percentage  on  contracts  completed  after  the  implementation  of 
the  FASA  and  Clinger-Cohen  Act. 

FF:  The  mean  final  cost  overrun  percentage  on  contracts  in  the  development  phase 
of  the  acquisition  life  cycle  before  the  implementation  of  the  FASA  and  Clinger- 
Cohen  Act  is  not  equal  to  the  mean  final  cost  overrun  percentage  on  contracts  in 
the  development  phase  after  the  implementation  of  the  FASA  and  Clinger-Cohen 
Act. 

H3:  The  mean  final  cost  overrun  percentage  on  contracts  in  the  production  phase 
of  the  acquisition  life  cycle  before  the  implementation  of  the  FASA  and  Clinger- 
Cohen  Act  is  not  equal  to  the  mean  final  cost  overrun  percentage  on  contracts  in 
the  production  phase  after  the  implementation  of  the  FASA  and  Clinger-Cohen 
Act. 

H4:  The  mean  final  cost  overrun  percentages  on  Army,  Navy,  and  Air  Force 
contracts  completed  before  the  implementation  of  the  FASA  and  Clinger-Cohen 
Act  are  not  equal  to  the  mean  final  cost  overrun  percentages  on  Army,  Navy,  and 
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Air  Force  contracts  completed  after  the  implementation  of  the  FASA  and  Clinger- 
Cohen  Act. 

H5:  The  mean  final  overrun  percentage  on  fixed  price  contracts  completed  before 
the  implementation  of  the  FASA  and  Clinger-Cohen  Act  is  not  equal  to  the  mean 
final  overrun  percentage  on  fixed  price  contracts  completed  after  the 
implementation  of  the  FASA  and  Clinger-Cohen  Act. 

He:  The  mean  final  overrun  percentage  on  cost  reimbursable  contracts  completed 
before  the  implementation  of  the  FASA  and  Clinger-Cohen  Act  is  not  equal  to  the 
mean  final  overrun  percentage  on  cost  reimbursable  contracts  completed  after  the 
implementation  of  the  FASA  and  Clinger-Cohen  Act. 

Hypotheses  two  and  three  were  motivated  by  Captain  David  Searle’s  thesis,  “The 
Impact  of  the  Packard  Commission’s  Recommendations  on  Reducing  Cost  Overruns  in 
Major  Defense  Programs.”  In  Captain  Searle’s  thesis,  he  shows  the  impact  of  the 
Packard  Commission’s  recommendations  on  cost  overruns  differed  between  the  stages  of 
the  acquisition  lifecycle.  Specifically,  he  found  that  cost  overruns  on  production 
contracts  experienced  no  change,  while  cost  overruns  on  development  contracts  worsened 
after  the  implementation  of  the  Packard  Commission’s  recommendations  (Searle,  1997: 
62-65). 

Similarly,  hypothesis  four  was  motivated  by  the  follow  up  article  to  Captain 
Searle’s  thesis.  The  sensitivity  analysis  performed  on  the  branches  of  service  showed 
that  Air  Force  contracts  worsened  after  the  implementation  of  the  Packard  Commission’s 
recommendations,  while  Army  and  Navy  contracts  experienced  no  change  (Christensen, 
1999:257) 
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Organization  of  the  Research 

This  study  essentially  picks  up  where  other  research  has  left  off  and  provides  an 
update  to  the  same  concerns  addressed  by  past  research.  The  next  chapter  will  examine 
more  closely  the  FASA  and  the  Clinger-Cohen  Act.  It  will  also  provide  a  more  detailed 
look  at  the  techniques  used  to  calculate  cost  overruns.  The  methodology  used  in  the 
research  is  detailed  in  the  third  chapter.  It  captures  the  data  collection  process  and  the 
methods  used  to  prove  or  disprove  the  hypotheses.  In  chapter  four,  the  results  of  the 
hypothesis  tests  are  summarized  and  discussed  along  with  the  statistical  tests  necessary  to 
show  the  tests  are  valid.  The  final  chapter  presents  the  conclusions  of  the  study  along 
with  recommendations  for  future  research. 
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II.  Literature  Review 


Chapter  Overview 

This  chapter  provides  the  reader  with  an  overview  of  the  acquisition  reform 
initiatives  relevant  to  this  research  and  a  discussion  of  the  tools  used  to  monitor  cost 
performance  in  the  Department  of  Defense  (DoD).  It  first  details  the  Federal  Acquisition 
Streamlining  Act  (FASA)  and  the  Clinger-Cohen  Act,  emphasizing  issues  that  are  most 
pertinent  to  cost  overruns.  Next,  the  Earned  Value  Management  System  (EVMS)  is 
discussed  along  with  how  the  EVMS  can  be  used  to  show  whether  or  not  a  program  has 
experienced  cost  overruns.  Finally,  the  chapter  concludes  with  a  discussion  of  past 
research  that  has  been  accomplished  with  regard  to  acquisition  reform  and  cost  overruns. 

The  Federal  Acquisition  Streamlining  Act 

On  October  13,  1994,  President  Clinton  signed  the  FASA  into  law.  The  FASA 
was  a  result  of  Vice  President  Gore’s  National  Perfonnance  Review  (NPR).  Prior  to  the 
FASA,  many  people  believed  the  acquisition  process  was  overly  cumbersome  and 
complex  (DSMC  web  site).  To  alleviate  those  problems,  the  FASA  introduced  broader 
and  more  encompassing  changes  to  the  acquisition  process  than  had  any  other  acquisition 
reform  legislation  in  the  10  years  preceding  (Tolan,  1998).  The  Simplified  Acquisition 
Threshold,  the  Federal  Acquisition  Computer  Network,  the  Truth  in  Negotiation  Act,  and 
military  standards  and  specifications  are  the  major  components  of  the  FASA  that  may 
have  an  effect  cost  performance. 
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The  Simplified  Acquisition  Threshold  (SAT)  establishes  a  ceiling  value  below 
which  procurement  officers  are  not  required  to  publicly  compete  commercial  buying 
activities.  The  FAS  A  raised  this  value  from  $25,000  to  $100,000.  In  order  for 
competition  not  to  be  circumvented,  the  FASA  stipulates  that  procurements  not  be  broken 
down  merely  to  avoid  requirements  that  apply  to  purchases  exceeding  the  SAT  (FAR  Part 
13).  It  also  eliminated  much  of  the  paperwork  that  goes  along  with  these  types  of 
purchases,  making  the  process  much  more  efficient  (Chinworth,  2000:  166). 

In  accordance  with  the  Federal  Acquisition  Regulation  (FAR),  purchases  between 
$2,500  and  $100,000  are  reserved  for  small  businesses  (FAR  Part  19).  Related  to  the 
small  business  concerns,  the  FASA  also  includes  preferential  treatment  for  small 
disadvantaged  businesses.  Specifically,  there  is  a  5%  goal  for  procurements  to  originate 
from  women-owned  businesses  (Tolan,  1998:  17).  Purchases  below  $2,500  are 
considered  “micropurchases”  and  are  exempt  from  the  small  business  requirement  and 
the  Buy  American  Act  (DSMC  web  site). 

The  Federal  Acquisition  Computer  Network  (FACNET)  was  introduced  with  the 
intentions,  per  the  FASA  objectives,  of  reducing  the  amount  of  paperwork  required 
within  the  acquisition  process  (DSMC  web  site).  The  FASA  introduced  the  FACNET  to 
the  entire  federal  government  (DSMC  web  site). 

Raising  the  threshold  at  which  requirements  of  the  Truth  in  Negotiation  Act 
(TINA)  apply  was  the  third  area  addressed  by  the  FASA  that  may  have  an  impact  on  cost. 
TINA  requires  contractors  to  submit  cost  and  pricing  data  and  to  certify  the  data  as 
accurate.  A  violation  of  TINA  could  result  in  defective  pricing,  and  the  contractor  could 
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be  penalized  (Northrop  Grumman,  2002).  The  FASA  raised  the  threshold  to  $500,000 
for  civilian  agencies  and  DoD  procurements  (DSMC  web  site). 

Perhaps  the  most  significant  change  brought  about  by  the  FASA  was  the 
elimination  of  many  military  standards  and  specifications  (DSMC  web  site).  Chinworth 
points  out  that  these  requirements  were  originally  designed  to: 

•  support  the  development  and  production  of  complex,  state-of-the-art  weapons 
systems  able  to  operate  in  demanding  environments  envisioned  by  military 
users; 

•  guarantee  that  long-term  logistic  requirements  could  be  met  for  complex 
weapons  systems; 

•  provide  consistency  in  the  implementation  of  complex  programs; 

•  guarantee  integrity  in  government  contracting  while  assuring  reasonable 
profitability  for  its  defense  contractors; 

•  maintain  public  accountability  and  prevent  contractor  abuses  (Chinworth, 
2000:  166). 

As  a  result  of  the  FASA,  many  procured  items  are  now  exempt  from  the 
requirement  that  cost  and  pricing  data  be  submitted  (DSMC  web  site).  Removing  the 
cost  and  pricing  data  submission  requirement  has  reduced  time  delays,  shortened  the 
research  and  development  process,  and  eliminated  many  design  specifications  and  tests. 
This  change  has  simplified  the  procurement  process  which  leads  to  cost  savings  (DSMC 
web  site). 

The  elimination  of  military  specifications  can  have  profound  cost  impact. 

Various  studies  address  the  idea  that  DoD  regulations  add  a  premium  to  the  cost  of 
products  and  services  provided  by  government  contractors.  In  1994,  one  such  study  was 
performed  by  TASC,  Inc.  and  Coopers  and  Lybrand.  It  has  since  become  known  as  the 
“Perry  Study”  (Chinworth,  2000:  167).  The  study  concluded  that  regulations  add  a 
significant  cost  premium  to  DoD  programs.  The  study  also  asserts  that  corrective  actions 
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can  be  accomplished  with  the  FASA,  and  no  other  statutory  changes  are  necessary 
(TASC,  1994:  47). 

A  major  improvement  in  reducing  the  number  of  DoD-specific  requirements  is 
the  implementation  of  Single  Process  Initiatives  (SPIs).  These  initiatives  give  contractors 
a  way  to  find  a  common  ground  among  many  production  programs  allowing  better 
utilization  of  commercial  standards  and  practices  (Chinworth,  2000:  167).  Specific  SPI 
proposals  originate  within  the  industry,  which  allow  the  contractor  to  develop  ideas  for 
making  the  processes  more  efficient  (Chinworth,  2000:  167).  SPI  proposals  are  tracked 
by  the  Defense  Contract  Management  Command  (DCMC)  via  the  Single  Process 
Initiative  System  (SPIS)  database,  which  is  composed  of  reports  from  contractors  and 
program  managers  (Chinworth,  2000:  168).  The  SPIS  database  categorizes  SPI 
initiatives  in  many  different  areas,  including  business,  engineering,  environmental, 
logistics,  manufacturing,  quality,  safety,  software,  and  testing  (Chinworth,  2000:  169). 
Some  of  the  SPI  initiatives  do  not  save  money  directly.  However,  those  initiatives  that 
have  reported  cost  savings  reveal  that  the  government  is  saving  money  through  this 
acquisition  reform  effort  (Chinworth,  2000:  171).  Table  1  shows  the  top  ten  cost 
reducing  processes  and  their  associated  savings. 


11 


Table  1.  Top  10  cost  reducing  processes  and  savings  (Chinworth,  2000:  170) 


Process  type 

No.  of  processes 
reporting  cost  data 

Combined  cost 
avoidance 

and  negotiated  savings 

Quality  systems 

45 

$51 M 

Testing 

16 

$42M 

Engineering-Configuration  Management 

43 

$38M 

Business  -  General 

29 

$34M 

Logistics  -  Parts/Material  Management 

16 

$25M 

Manufacturing  -  Soldering/Welding 

35 

$24M 

Quality  -  Multiple  Processes 

38 

$19M 

Business  -  Earned  Value  Management  System 

13 

$16M 

Quality  -  Nonconforming  Material/MRB 

13 

$15M 

Quality  -  Inspection 

14 

$1 1M 

The  Clinger-Cohen  Act 

Passed  in  1996,  the  Clinger-Cohen  Act,  named  for  Congressman  William  Clinger 
and  Senator  William  Cohen,  resulted  from  two  other  pieces  of  acquisition  refonn 
legislation,  the  Federal  Acquisition  Reform  Act  (FARA)  and  the  Information  Technology 
Management  Reform  Act  (ITMRA).  The  FARA  and  the  ITMRA  were  passed  as  separate 
initiatives,  since  the  impact  they  had  on  each  other  made  them  difficult  to  pass  together 
(FISC  website).  These  two  acts  further  advanced  the  changes  made  by  the  FASA 
(DSMC  website).  Each  is  addressed  separately  in  the  following  discussion. 

The  FARA  made  specific  changes  to  the  Simplified  Acquisition  Procedures 
(SAP).  The  FARA  requires  that  SAP  be  used  for  commercial  item  purchases  between 
$100,000  and  $5  million  (Pendolino  and  Causey,  1996:  7).  The  FARA  also  removed  the 
link  between  SAP  and  the  FACNET.  As  mentioned  earlier,  the  FASA  had  raised  the 
simplified  acquisition  threshold  from  $25,000  to  $100,000.  It  also  required,  however, 
that  agencies  adhere  to  a  $50,000  threshold  until  they  had  implemented  FACNET.  The 
FARA  repealed  this  requirement,  but  stipulated  that  the  agency  must  have  full  FACNET 
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capability  by  the  end  of  1999  or  its  threshold  would  be  reduced  to  $50,000  (Pendolino 
and  Causey,  1996:  7). 

Also  addressed  in  the  FARA,  commercial  items  are  exempt  from  the  TINA.  That 
is,  vendors  are  not  required  to  submit  cost  or  pricing  data  for  contracts  below  the  SAT  of 
$500,000  (Pendolino  and  Causey,  1996:  8).  The  FARA  also  released  commercial  items 
from  the  requirements  of  the  Cost  Accounting  Standards  (CAS).  Commercial  items  now 
have  their  own  guidance  consistent  with  commercial  practices,  FAR  cost  principles,  and 
CAS.  This  guidance  is  intended  to  ensure  contractors  do  not  overcharge  the  government 
(Pendolino  and  Causey,  1996:  8). 

The  second  piece  of  legislation  comprising  the  Clinger-Cohen  Act  is  the 

Information  Technology  Management  Refonn  Act  (ITMRA),  passed  because  of  long 

delays  and  multi-million  dollar  cost  overruns  on  infonnation  technology  (IT)  projects. 

Two  prime  examples  were  the  Internal  Revenue  Service  (IRS)  tax  systems  modernization 

and  the  Federal  Aviation  Administration  (FA A)  advanced  automation  system 

(Beachboard,  1997).  The  ITMRA  states  that  the  FAR  council  will  ensure  that, 

to  the  maximum  extent  practicable,  the  process  for  acquisition  of 
information  technology  is  a  simplified,  clear,  and  understandable  process 
that  specifically  addresses  the  management  of  risk,  incremental 
acquisitions,  and  the  need  to  incorporate  commercial  information 
technology  in  a  timely  manner  (ITMRA  Sec.  5201). 

One  of  the  biggest  changes  brought  about  by  the  ITMRA  is  the  repeal  of  Section 
1 1 1  of  the  Federal  Property  and  Administrative  services  Act  of  1949  known  as  the 
Brooks  Act.  The  Brooks  Act  gave  the  General  Services  Administration  exclusive 
authority  to  procure  information  technology  resources  for  all  of  the  federal  government 
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(Pendolino  and  Causey,  1996:  12).  The  ITMRA  effectively  gives  control  of  IT  resources 
back  to  the  individual  agencies. 

Oversight  for  the  acquisition  of  IT  resources  rests  with  the  Director  of  the  Office 
of  Management  and  Budget  (OMB).  The  director  is  required  to  encourage  agencies  to 
procure  IT  resources  in  a  cost-effective  manner  (Pendolino  and  Causey,  1996:12).  Along 
with  the  responsibility  of  procuring  IT  resources  for  their  agency,  each  executive  director 
must  issue  guidance  concerning  cost/benefits,  risks,  and  other  evaluation  criteria  for  IT 
acquisitions  (Pendolino  and  Causey,  1996:  13). 

The  ITMRA  introduced  a  concept  known  as  modular  contracting  for  IT  resources. 
Under  this  concept,  agencies  acquire  an  IT  system  in  successive  increments,  which 
allows  them  to  manage  large  acquisitions  more  efficiently.  The  ITMRA  requires  the 
FAR  to  contain  guidance  on  the  use  of  modular  contracting  (Pendolino  and  Causey, 

1996:  13).  The  ITMRA  also  encourages  agencies  to  use  pilot  programs  that  test  new  and 
innovative  ways  of  acquiring  IT  resources.  Upon  discovering  a  way  that  saves  money, 
the  agency  and  contractor  are  allowed  to  share  the  savings  (Pendolino  and  Causey,  1996: 
13). 

The  Earned  Value  Management  System 

This  section  discusses  the  building  blocks  of  the  earned  value  management 
system  (EVMS)  and  how  it  is  used  to  calculate  cost  overruns.  The  EVMS  has  its  roots  in 
industry,  but  is  primarily  used  by  the  DoD  to  analyze  defense  contracts  and  detennine  if 
they  are  on  track  with  respect  to  their  proposed  budget  and  schedule  (Christensen,  1998: 
374).  The  government  first  mandated  the  use  of  earned  value  in  1967  as  part  of  the 
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Cost/Schedule  Control  Systems  Criteria  (C/SCSC)  (Fleming  and  Koppelman,  2000:  59). 
In  1996,  the  DoD  changed  the  name  from  C/SCSC  to  the  Earned  Value  Management 
Control  Systems  Criteria  (DoD,  1996). 

For  the  EVMS  to  be  effective,  projects  must  have  certain  elements.  A  budget  and 
a  schedule  are  two  of  the  most  important  elements.  These  elements  provide  a  clear, 
demonstrable  method  for  managing  a  defense  contract  of  any  size  (Christensen,  1998: 
375).  Projects  are  typically  subdivided  into  smaller,  more  manageable  elements  or  work 
packages.  These  work  packages  were  originally  called  Cost  Account  Plans,  but  were 
more  aptly  renamed  in  1996  as  Control  Account  Plans  (CAP),  since  more  than  costs 
alone  are  controlled  at  this  level  (Fleming  and  Koppelman,  2000:  59).  Each  CAP 
comprises  a  detailed  work  plan,  a  time  to  start  and  finish  each  task,  and  an  authorization 
for  resources  (i.e.  budget).  The  sum  of  all  the  CAPs  makes  up  a  project’s  scope, 
schedule,  and  budget  (Fleming  and  Koppelman,  2000:  111). 

There  are  three  fundamental  building  blocks  of  the  EVMS  from  which  most 
earned  value  calculations  can  be  accomplished.  These  three  building  blocks  include  the 
Budgeted  Cost  of  Work  Scheduled  (BCWS),  the  Budgeted  Cost  of  Work  Performed 
(BCWP),  and  the  Actual  Cost  of  Work  Perfonned  (ACWP)  (DSMC  website).  The 
glossary  of  earned  value  terms  from  the  website  of  the  Office  of  the  Under  Secretary  of 
Defense  for  Acquisition  Technology  and  Logistics  gives  the  following  definitions  for 
each  of  these  terms: 

1 .  BCWS  -  the  sum  of  budgets  for  all  work  packages  and  the  amount  of 
effort  scheduled  to  be  completed  by  a  certain  time.  It  is  used  primarily  for 
calculations  related  to  schedule  and  will  not  be  used  for  the  purpose  of  this 
research. 
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2.  BCWP  -  the  sum  of  budgets  for  completed  work  packages  and  completed 
portions  of  work  packages.  This  is  the  “earned  value”  part. 

3.  ACWP  -  the  costs  that  were  actually  incurred  and  recorded  in 
accomplishing  the  work  in  a  given  time  period. 

The  Budget  at  Completion  (BAC)  and  the  Estimate  at  Completion  (EAC)  are  two 
other  EVMS  tools  necessary  in  calculating  cost  overruns.  The  BAC  is  the  sum  of  all 
budgets  established  for  the  contract  (DAU,  2001).  As  Figure  2  shows  below,  it  is  the 
BCWS  line  extended  to  the  end  of  the  time  period  for  any  given  contract.  The  EAC  is 
more  difficult  to  calculate,  since  there  are  a  multitude  of  EAC  formulas.  For  this  reason, 
contractors  usually  submit  a  range  of  EACs  (Christensen,  1993:  17).  One  of  the  most 
popular  EAC  computations  uses  the  cumulative  Cost  Performance  Index  (CPI)  as  the 
basis  for  determining  the  EAC  (Christensen,  1996:  37).  The  cumulative  CPI  (CPIcum)  is 
the  BCWP  to  date  divided  by  the  ACWP  to  date.  A  CPICUm  less  than  one  is  undesirable 
and  indicates  the  contract  is  overrunning  costs  (Christensen,  2001).  Figure  2  shows  the 
EAC  as  the  ACWP  line  extrapolated  out  to  the  end  of  the  time  period  for  any  given 
contract. 

The  cumulative  Cost  Variance  (CV)  is  calculated  by  subtracting  the  ACWP  from 
the  BCWP  (DSMC  website).  A  negative  number  is  the  cost  overrun  to  date;  a  positive 
number  indicates  the  contract  is  under  running  costs.  The  Variance  at  Completion  (VAC) 
is  calculated  by  subtracting  the  EAC  from  the  BAC  (DSMC  website).  Similar  to  the  CV, 
a  negative  VAC  indicates  a  final  cost  overrun  while  a  positive  number  indicates  no  cost 
overrun.  See  Figure  2  for  a  graphical  representation  of  these  calculations. 
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Figure  2.  EVMS  Graph 

Past  Research  Regarding  Acquisition  Reform  and  Cost  Overruns 

This  section  discusses  past  research  related  to  acquisition  refonn  and  cost 
overruns.  The  discussion  is  organized  to  illustrate  how  this  research  complements  these 
past  efforts.  It  explains  the  focus,  methodology,  and  conclusions  of  the  following  studies: 
“The  Impact  of  the  Packard  Commission’s  Recommendations  on  Reducing  Cost 
Overruns  on  Defense  Acquisition  Contracts”  (Christensen,  Searle,  and  Vickery,  1999), 
“Cost  Overrun  Optimism:  Fact  or  Fiction?”  (Christensen,  1994),  “Is  the  CPI-Based  EAC  a 
Lower  Bound  to  the  Final  Cost  of  Post  A-12  Contracts?”  (Christensen  and  Rees, 2001), 
and  “Does  a  Rubber  Baseline  Guarantee  Cost  Overruns  on  Defense  Acquisition 
Contracts”  (Christensen  and  Gordon,  1998). 
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The  Packard  Commission  was  formed  by  President  Ronald  Reagan  in  1986  to 
review  the  defense  acquisition  process  and  recommend  improvements  to  the  procurement 
process  that  would  minimize  cost  growth  and  reduce  schedule  delays  (Christensen  et  ah, 
1999:  25 1).  In  their  study  of  the  Packard  Commission,  Christensen,  Searle,  and  Vickery 
analyzed  the  effect  of  the  recommendations  by  the  Packard  Commission  on  cost  overruns 
among  defense  contracts  (Christensen  et  ah,  1999:  251). 

Prior  to  data  analysis,  Christensen,  Searle,  and  Vickery  hypothesized  that  cost 
performance  among  defense  contracts  would  improve  (i.e.,  cost  overruns  would  decrease) 
after  implementation  of  the  Packard  Commission’s  recommendations  (Christensen  et  ah, 
1999:  254).  They  tested  this  hypothesis  by  comparing  the  average  cost  overrun 
percentage  on  contracts  completed  before  implementation  to  the  average  cost  overrun 
percentage  on  contracts  completed  after  implementation  using  a  non-parametric  test 
(Christensen  et  ah,  1999:  254).  They  used  a  non-parametric  test  because  a  parametric  test 
failed  the  assumptions  of  normality  and  constant  variance  (Christensen,  1999:  262). 

They  also  tested  whether  cost  performance  was  sensitive  to  contract  phase  or  branch  of 
service  (Christensen  et  al.,  1999:  256).  Using  269  contracts  from  the  DAES  database, 
their  study  yielded  the  results  shown  in  Table  2. 
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Table  2.  The  Effect  of  Packard  Commission  Recommendations  on  Defense  Cost 
Performance  (Christensen  et  al.,  1999:  257) 


Contract  Phase 

Managing  Service 

All 

Contracts 

Development 

Contracts 

Production 

Contracts 

Navy 

Army 

269 

8 

188 

113 

134 

22 

Hnal  overrun  betore 
implementation  (%) 

5.6 

4.1 

6.2 

2.8 

7.6 

8.1 

Final  overrun  atter 
implementation  (%) 

9.5 

15.3 

7.2 

12.7 

6.1 

17 

3.9 

11.2 

1.0 

9.9 

-1.5 

8.9 

0.055 

0.014 

0.294 

0.003 

0.206 

0.110 

At  an  alpha  of  0.10,  they  found  that  cost  performance  worsened  overall  after 
implementation  of  the  Packard  Commission’s  recommendations  (Christensen  et  al.,  1999: 
256).  As  Table  2  shows,  they  also  discovered  that  the  change  in  cost  performance  was 
sensitive  to  contract  phase  and  branch  of  service.  The  effect  of  implementing  the 
Packard  Commission’s  recommendations  varied  between  phases  and  branches  of  service. 
Perhaps  the  most  startling  finding  from  their  study  was  that  cost  overruns  on  Air  Force 
contracts  more  than  tripled  after  implementation  (Christensen  et  al.,  1999:  257). 

The  method  used  to  calculate  Estimates  at  Completion  (EAC)  is  pertinent  to  this 
research.  Seeing  how  EACs  are  computed  helps  the  reader  see  why  there  is  a  systematic 
problem  with  government  contracts  overrunning  costs.  “Cost  Overrun  Optimism:  Fact  or 
Fiction?”  (Christensen)  is  one  study  that  shows  how  EACs  are  computed.  In  this  study, 
Christensen  demonstrates  the  existence  of  a  cultural  problem  by  proving  that  cost 
overruns  projected  by  the  contractor  and  government  are  too  optimistic  over  the  life  of 
the  contract  (Christensen,  1994:  25). 

Christensen’s  1994  study  was  partially  motivated  by  the  A-12  Administrative 
Inquiry  completed  by  Chester  Beach,  which  found  “abiding  cultural  problems”  not 
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unique  to  the  Navy  (Christensen  1996:  36).  In  this  inquiry,  Beach  speculates  that 
pessimistic  EACs  (i.e.,  estimates  that  make  the  program  look  bad)  are  held  back  to 
protect  the  program  and  program  managers’  careers  (Christensen,  1994:  25).  Using  64 
completed  contracts,  Christensen  tested  this  assertion  by  comparing  overruns  at  various 
completion  points  with  projected  final  costs  to  determine  if  government  and  contractor 
estimates  are  consistently  optimistic  (Christensen,  1994:  25).  For  the  purpose  of  his 
study,  Christensen  defined  an  overly  optimistic  estimate  as  one  in  which  the  projected 
final  overrun  is  less  than  the  current  overrun.  As  justification  for  this  definition, 
Christensen  cites  multiple  studies  supporting  the  conclusion  that  a  contract’s  cost 
performance  will  not  improve  after  the  15%  completion  point  (Christensen,  1994:  29). 

He  found  that  the  government’s  estimated  overrun  and  the  contractor’s  estimated  overrun 
were  consistently  optimistic  throughout  the  life  of  contract. 

Another  study,  “Is  the  CPI-Based  EAC  a  Lower  Bound  to  the  Final  Cost  of  Post 
A- 12  Contracts?”  (Christensen  and  Rees,  2001)  further  enhances  the  understanding  of 
how  an  EAC  is  calculated.  In  this  study,  Christensen  and  Rees  prove  the  validity  of  using 
the  government’s  rule  for  using  the  EAC  based  on  the  cumulative  CPI  (EACcpi)  as  a  floor 
to  the  final  cost  of  a  contract  (Christensen  and  Rees,  2001).  In  other  words,  using  the 
EACCpi  assumes  the  cost  performance  of  a  contract  will  not  improve  and  will  either  stay 
the  same  or  worsen  throughout  the  life  of  the  contract.  To  test  the  rule,  Christensen  and 
Rees  formed  the  following  null  hypothesis: 

H0:  CPI-based  EAC  >  Final  Cost 

If  the  null  hypothesis  is  not  rejected,  the  EACcpi  is  not  a  floor  to  the  final  cost.  Rejection 
of  the  null  validates  the  rule  of  using  the  EACcpi.  For  the  purpose  of  this  study,  the 
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cumulative  CPI  was  calculated  at  three  different  completion  points:  the  20%  point,  the 
50%  point,  and  the  70%  point  (Christensen  and  Rees,  2001). 


Since  the  Defense  Acquisition  Executive  Summary  database  has  contracts  of  all 
sizes,  Christensen  and  Rees  adjusted  for  these  differences  by  normalizing  the  data  as  the 

EAC  -  Final  Cost 


deviation  from  final  cost  (DAC)  using  the  following  formula:  DAC  = 


Final  Cost 


(Christensen  and  Rees,  2001).  Using  52  post  A- 12  contracts  (i.e.,  contracts  starting  after 
3 1  December  1991),  they  subdivided  the  database  into  contracts  in  development  versus 
contracts  in  production,  cost  reimbursable  contracts  versus  fixed  price  contracts,  and 
Army  contracts  versus  Navy  contracts  versus  Air  Force  contracts.  The  results  are  shown 
Table  3. 


Table  3.  EAC  Deviation  From  Final  Cost  (Christensen  and  Rees,  2001) 


Contract 

Category 

n 

Early  Stage 

Middle  Stage 

Late  Stage 

mean 

P 

mean 

P 

mean 

p  ! 

All 

52 

-0.107 

0.001 

-0.047 

0.014 

0.003 

0.559 

Development 

24 

-0.146 

0.003 

-0.069 

0.029 

0.001 

0.509 

Production 

26 

-0.073 

0.041 

-0.018 

0.23 

0.018 

0.834 

Cost  Reimbursable 

34 

-0.144 

0.001 

-0.063 

0.018 

0.002 

0.527 

Fixed  Price 

15 

-0.025 

0.255 

-0.019 

0.26 

0.008 

0.718 

Army 

14 

-0.112 

0.04 

-0.076 

0.077 

0.014 

0.615 

Air  Force 

12 

-0.185 

0.011 

-0.052 

0.121 

0.003 

0.532 

Navy 

26 

-0.069 

0.057 

-0.028 

0.135 

-0.004 

0.575 

As  Table  3  shows,  at  an  alpha  of  0.05,  the  EACcpi  is  a  reasonable  floor  to  the  final  cost  of 


contracts  in  the  early  and  middle  stages  for  all  but  Fixed  Price  and  Navy  contracts. 


When  a  contract  undergoes  frequent  changes,  many  people  assume  the  contract  is 


more  vulnerable  to  a  cost  overrun.  Christensen  and  Gordon’s  study,  “Does  a  Rubber 


Baseline  Guarantee  Cost  Overruns  on  Defense  Acquisition  Contracts,”  addresses  this 


notion.  When  changes  are  large  and  occur  frequently,  the  contract  has  what  is  known  as 
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a  “rubber  baseline”  and  is  thought  by  some  people  to  be  the  root  of  cost  overruns 
(Christensen  and  Gordon,  1998:  43). 

Christensen  and  Gordon  tested  the  relationship  between  cost  overruns  and 
baseline  stability  using  two  statistical  models.  The  first-test  was  a  two  factor  analysis  of 
variance  (ANOVA)  test  using  the  schedule  cost  index  (SCI)  as  the  measure  of  cost 
overruns.  The  primary  factor  was  the  number  of  changes  to  the  contract  using  the 
contract  budget  base  (CBB)  as  the  measure  of  baseline  stability.  The  CBB  includes  the 
Budget  at  Completion  (BAC)  and  the  Management  Reserve  (MR).  A  significant  change 
is  defined  as  one  that  changes  the  CBB  by  more  than  10%.  The  secondary  factor  used  in 
the  ANOVA  test  was  the  contract’s  completion  point.  To  test  the  relationship  between 
the  two  factors  and  the  SCI,  they  used  an  alpha  level  of  0.05  (Christensen  and  Gordon, 
1998:  45). 

The  second  analysis  accomplished  was  a  linear  regression  test  using  the  last 
reported  CPI  (CPIl)  as  the  measure  of  cost  overruns  and  the  coefficient  of  variation  of  the 
Performance  Measurement  Baseline  (CVpmb)  as  the  measure  for  baseline  stability. 
Statistical  significance  was  determined  using  the  regression  coefficient  of  the  CVpmb-  If 
the  coefficient  of  the  CVpmb  was  significantly  different  from  zero,  they  concluded  that 
there  is  a  relationship  between  the  CPI  and  the  PMB  (Christensen  and  Gordon,  1998:  46). 

Christensen  and  Gordon’s  study  using  the  DAES  database  included  669  contracts 
from  165  programs  for  the  ANOVA  test  and  401  contracts  encompassing  131  programs 
for  the  linear  regression  test.  Based  on  the  results  from  both  tests,  they  found  that  there 
was  no  statistical  relationship  between  baseline  stability  and  cost  overruns.  Also,  this 
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finding  was  not  sensitive  to  the  branch  of  service  in  question  (Christensen  and  Gordon, 
1998:  47).  A  summary  of  their  findings  from  each  test  is  shown  in  Tables  4  and  5  below. 


Table  4.  Summary  of  ANOVA  Test 


Sources  of  Variation 

Sums  of  Squares 

Degrees  of 
Freedom 

Mean  Square 

F 

Factor  A:  Quarters 

0.1099 

3 

0.0366 

1.251 

2.618 

Factor  B:  Changes 

0.0890 

4 

0.0223 

0.760 

2.385 

Interaction 

0.5914 

12 

0.0493 

1.683 

1.767 

Error 

19.561 

668 

0.0293 

Table  5.  Summary  of  Regression  Analysis 


Sample 

CPIL 

cvPMB 

Slope 

Category 

n 

Mean 

Std  Dev 

Mean 

Std  Dev 

Pi 

Std  Error 

t 

AF 

151 

0.96 

0.11 

0.36 

0.58 

0.004 

0.015 

0.271 

Navy 

142 

0.94 

0.11 

0.32 

0.42 

-0.009 

0.219 

-0.407 

Army 

108 

0.92 

0.12 

0.62 

0.72 

-0.006 

0.016 

-0.364 

The  results  of  Christensen  and  Gordon’s  study  showed  that  cost  overruns  were  not 
caused  by  the  stability  of  the  baseline.  They  conclude  their  study  by  stating  that  more 
research  was  needed  in  detennining  what  caused  a  contract  to  have  a  cost  overrun.  While 
this  thesis  does  not  directly  address  the  cause  of  cost  overruns,  it  does  provide  further 
insight  into  the  issues  addressed  by  the  FASA  and  the  Clinger-Cohen  Act  as  they  relate  to 
cost  overruns.  This  insight  can  be  used  to  stimulate  further  research  into  the  issues 
addressed  by  the  two  pieces  of  legislation. 
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III.  Methodology 


Chapter  Overview 

This  chapter  provides  an  overview  of  the  process  and  tools  used  to  conduct  the 
research  and  data  analysis  for  this  effort.  It  begins  with  a  discussion  of  the  Defense 
Acquisition  Executive  Summary  (DAES)  database,  followed  by  how  its  data  is 
aggregated  into  the  sample  that  is  used  for  testing  the  hypotheses.  It  then  re-introduces 
the  hypotheses  stated  in  Chapter  1  coupled  with  a  more  thorough  discussion  of  each 
hypothesis  and  how  each  pertains  to  the  research.  The  hypotheses  section  is  followed  by 
a  presentation  of  the  tests  that  will  be  used  to  prove  or  disprove  the  hypotheses.  This 
section  will  define  the  hypothesis  tests  and  their  corresponding  assumptions  and  also 
describe  the  tests  associated  with  each  assumption.  The  chapter  concludes  by  detailing 
the  significance  levels  and  possible  errors  that  could  occur. 

DAES  Database 

The  DAES  database  is  a  part  of  the  Consolidated  Acquisition  Reporting  System 
(CARS),  which  is  a  “personal  computer-based  data  entry  and  reporting  system”  (CARS 
website).  CARS  is  comprised  of  the  DAES,  the  Selected  Acquisition  Report  (SAR),  and 
the  Acquisition  Program  Baseline  (APB).  Its  main  purpose  is  to  achieve  the  following 
objectives: 

1 .  Reduce  the  workload  in  preparing  the  APB,  DAES,  and  SAR 

2.  Standardize  and  automate  project  status  information  for  program  managers 
and  other  acquisition  executives 
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3.  Provide  timely  and  accurate  tools  to  the  Office  of  the  Under  Secretary  of 
Defense  (Acquisition,  Technology,  and  Logistics)  (OUSD(AT&L))  and 
the  military  services  for  the  purpose  of  analysis 

4.  Improve  acquisition  data  management  and  interchange  capabilities 

5.  Establish  software,  data,  and  documentation  standards  for  acquisition 
(CARS  website). 

The  use  of  CARS  is  mandatory  for  all  Major  Defense  Acquisition  Programs  (AC AT  I) 
and  Major  Automated  Information  Systems  (AC  AT  IA).  An  AC  AT  I  program  is  defined 
as  a  program  estimated  by  the  Under  Secretary  of  Defense- Acquisition  Technology  and 
Logistics  (USD-AT&L)  to  require  eventual  expenditure  for  research,  development,  test, 
and  evaluation  of  more  than  $365  million  in  Fiscal  Year  (FY)  2000  constant  dollars.  A 
program  is  considered  an  AC  AT  IA  if  it  is  1)  designated  by  the  Assistant  Secretary  of 
Defense  for  Command,  Control,  Communications,  and  Intelligence  (ASD-C3I),  or  2) 
estimated  to  require  more  than  $32  million  in  program  costs  for  any  given  year,  more 
than  $126  million  in  total  program  costs,  or  more  than  $378  million  in  total  lifecycle 
costs.  All  costs  are  FY  2000  constant  dollars  (DAU  Glossary,  2001).  The  CARS 
software  specifies  the  fonnat  of  the  DAES  with  the  exception  of  the  memo  and  comment 
fields  (DOD  5000.2-R,  2002). 

The  DAES  serves  as  an  early  warning  report  to  the  OUSD-AT&L  by  describing 
any  actual  and  potential  program  problems.  It  also  describes  any  mitigating  actions  taken 
(DOD  5000.2-R,  2002).  The  DAES  presents  total  projected  costs  and  quantities  for  all 
years  through  the  end  of  the  current  phase  of  the  acquisition  life  cycle  (DOD  5000.2-R, 
2002).  All  ACAT  I  and  ACAT  IA  programs  are  required  to  report  program  status 
information  to  the  DAES  database  on  a  quarterly  basis  (DOD  5000.2-R,  2002).  There  are 
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two  times  when  this  quarterly  cycle  is  not  adhered  to.  It  is  not  adhered  to  when  a  Nunn- 
McCurdy  breach  has  occurred  or  is  suspected  to  occur.  A  Nunn-McCurdy  breach  is 
when  “the  program  acquisition  unit  cost  (PAUC)  or  the  average  procurement  unit  cost 
(APUC)  (in  base-year  dollars)  increases  by  15%  or  more  over  the  PAUC  objective  or 
APUC  objective  of  the  currently  approved  APB  (in  base  year  dollars)”  (DOD  5000.2-R, 
2002).  The  DAES  quarterly  reporting  cycle  is  also  not  adhered  to  when  a  Program 
Objective  Memorandum  (POM)  or  Budget  Estimate  Submission  (BES)  causes  a  program 
to  deviate  from  the  approved  APB  threshold.  Both  of  these  situations  require  an  out  of 
cycle  submission  to  the  DAES  database  (DOD  5000.2-R,  2002). 

The  following  columns  from  the  DAES  database  are  pertinent  to  this  research: 

1 .  Contract  Identification  (CID)  -  The  CID  is  a  number  assigned  to  each 
contract.  It  is  used  in  this  research  to  ensure  that  no  contracts  are  repeated 
and  to  identify  the  branch  of  service  responsible  for  the  contract. 

2.  Contract  Number  (CNO)  -  The  CNO  is  a  number  assigned  to  each 
contract  for  further  identification.  It  is  also  used  in  this  research  to 
determine  that  no  contracts  are  repeated. 

3.  Submit  Date  (SUBMITDATE)  -  This  is  the  date  the  data  was  submitted  to 
the  DAES  database.  It  is  used  in  this  research  to  represent  the  contract 
completion  date. 

4.  Program  Phase  (PPHASE)  -  PPHASE  is  the  phase  of  the  acquisition 
lifecycle  the  contract  was  in  at  the  time  the  data  was  submitted.  It  is  used 
in  this  research  to  divide  the  data  into  phases. 

5.  Actual  Cost  of  Work  Performed  (ACWP)  -  An  element  of  the  EVMS,  it  is 
the  cumulative  cost  of  the  work  completed  to  date.  It  is  used  in  this 
research  to  calculate  cost  overruns. 

6.  Budgeted  Cost  of  Work  Performed  (BCWP)  -  The  BCWP  is  the 
cumulative  budgeted  cost  of  the  work  perfonned  to  date.  It  is  also  used  in 
this  research  to  calculate  cost  overruns. 
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7.  Cumulative  Budget  Base  (CBBASE)  -  The  CBBASE  is  the  total  budget  for 
the  contract.  It  is  used  in  conjunction  with  the  Management  Reserve  to 
calculate  the  completion  percentage  of  a  contract. 

8.  Management  Reserve  (MR)  -  The  MR  is  money  set  aside  that  is  used  at 
the  discretion  of  top  management. 

9.  Contract  Type  (CTYPE)  -  The  CTYPE  is  the  type  of  contract  for  the 
corresponding  line  item.  It  is  used  to  subdivide  the  database  into  Fixed 
Price  contracts  and  Cost  Reimbursable  contracts. 

Since  the  DAES  database  contains  contracts  dated  back  to  1970,  it  had  to  be 
scaled  back  to  include  only  those  contracts  relevant  to  the  research  questions.  First,  all 
contracts  completed  before  December  31,  1993  and  after  December  31,  2001  were 
removed.  This  time  period  was  selected  based  on  Searle’s  Packard  Commission  study 
(Searle,  1997:  39).  The  time  period  was  also  based  on  expert  opinion  after  discussions 
with  Dr.  David  Christensen,  an  expert  in  the  earned  value  field  and  Ms.  Sandra  Meckley, 
a  representative  of  the  acquisition  office  for  the  Secretary  of  the  Air  Force.  This 
timeframe  is  appropriate  since  it  encompasses  the  enacting  of  the  FASA  and  the  Clinger- 
Cohen  Act.  Contracts  within  this  timeframe  are  further  separated  into  groups  on  either 
side  of  December  31,  1997. 

The  second  step  in  organizing  the  DAES  database  used  the  following  equation  to 
determine  the  percentage  completion  of  each  contract: 

BCWP 

CBBASE -MR 

Contracts  that  were  not  more  than  75%  complete  were  removed  from  the  database.  Most 
costs  are  incurred  between  the  15  and  75  percent  completion  points.  This  is  shown  in 
Figure  2  by  the  cost  curve  flattening  out  as  the  contract  nears  completion.  Studies  have 
shown  costs  to  be  relatively  stable  beyond  the  75%  completion  point.  Thus,  a  contract 
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that  is  at  least  75%  complete  is  considered  complete  in  terms  of  costs  incurred  (Searle, 
1997:41). 


The  third  step  used  the  CID  and  the  CNO  to  detennine  if  there  were  multiple 
instances  of  the  same  contract.  If  any  duplicate  contracts  were  found,  the  latest 
submission  was  retained  and  the  rest  were  discarded.  The  following  table  shows  the 
number  of  contracts  in  each  sample  that  were  used  for  hypotheses  testing. 


Table  6.  Treatment  Matrix  of  DOD  Contracts 


Category 

P  re- 
1997 

Post-1997 

Ff  -  Total  Contracts 

107 

113 

H2  -  Development 

41 

64 

H3  -  Production 

66 

49 

H4  -  Army 

19 

28 

H4  -  Navy 

47 

58 

H4  -  Air  Force 

41 

27 

H5  -  Fixed  Price 

39 

54 

H6  -  Cost 

Reimbursable 

67 

59 

Research  Questions  and  Hypotheses 

This  section  reintroduces  the  research  questions  posed  in  Chapter  1 .  Each 
research  question  will  be  followed  by  a  corresponding  hypothesis  and  a  brief  discussion 
of  its  relevance  to  the  research.  Keep  in  mind  that  the  overall  goal  of  this  research  is  to 
determine  if  the  FASA  and  the  Clinger-Cohen  Act  had  any  effect  on  cost  overruns  among 
DoD  contracts. 

Research  Question  #1 

Compared  to  contracts  completed  before  the  FASA  and  Clinger-Cohen  Act,  did 
cost  overruns  on  contracts  completed  after  the  FASA  and  Clinger-Cohen  Act 
significantly  worsen,  stay  the  same,  or  improve? 


28 


Research  Hypothesis  #1 

Ha:  The  mean  final  cost  overrun  percentage  on  contracts  completed  before 
December  31,  1997  is  not  equal  to  the  mean  final  cost  overrun  percentage  on 
contracts  completed  after  December  31,  1997. 

The  answer  to  this  question  will  help  determine  whether  the  FASA  and  the 
Clinger-Cohen  Act  had  an  effect  on  cost  overruns.  The  final  cost  overrun  percentage  is 
used  instead  of  the  actual  overrun  to  make  the  contracts  comparable.  The  DAES  includes 
contracts  of  all  sizes,  and  to  compare  a  $1  million  contract  to  a  $100  million  contract 
could  be  misleading.  To  illustrate,  a  $1  million  cost  overrun  on  the  former  contract 
would  yield  an  overrun  percentage  of  100%,  whereas  the  same  cost  overrun  on  the  latter 
contract  would  be  a  1%  overrun.  Thus,  the  overruns  are  normalized  to  prevent  this 
problem.  Equation  1  presented  below  is  used  to  calculate  the  final  cost  overrun. 

Equation  2  is  used  to  convert  the  final  overrun  from  a  dollar  amount  to  a  percentage.  The 
Budget  at  Completion  (BAC)  is  calculated  by  subtracting  the  Management  Reserve  (MR) 
from  the  Contract  Budget  Base  (CBBASE). 

FinalOverrun  =  BAC  -  ACWP  (1) 


n/  BAC -ACWP  . 

r  iTiuluvemui/o  — -  (2) 

BAC 


Research  question  #2  is  addressed  by  two  hypotheses.  They  are  presented  together  with 
the  discussion  following. 


Research  Question  #2 


Is  the  effect  of  the  FASA  and  Clinger-Cohen  Act  on  cost  overruns  sensitive  to  the 
phase  of  the  acquisition  life  cycle  the  contract  is  in? 
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Research  Hypothesis  #2 

Ha:  With  respect  to  contracts  in  the  development  phase  of  the  acquisition 
lifecycle,  the  final  overrun  percentage  on  contracts  completed  before  December 
31,  1997  is  not  equal  to  the  final  overrun  percentage  on  contracts  completed  after 
December  31,  1997. 

Research  Hypothesis  #3 

HA:  With  respect  to  contracts  in  the  production  phase  of  the  acquisition  lifecycle, 
the  final  overrun  percentage  on  contracts  completed  before  December  31,  1997  is 
not  equal  to  the  final  overrun  percentage  on  contracts  completed  after  December 
31,1997. 


After  considering  the  overall  effect  the  FASA  and  Clinger-Cohen  Act  have  had  on 
cost  overruns,  the  sensitivity  to  phases  of  the  acquisition  lifecycle  is  addressed  by  the  two 
research  questions  stated  above.  Research  hypothesis  #2  looks  at  development  contracts, 
while  research  hypothesis  #3  deals  with  only  those  contracts  in  the  production  phase  of 
the  acquisition  lifecycle.  Even  if  this  research  determines  the  FASA  and  Clinger-Cohen 
Act  have  had  no  effect  on  cost  overruns,  the  answer  to  these  questions  will  provide  senior 
leadership  with  additional  insight  as  to  where  efforts  should  be  focused  on  reducing  cost 
overruns.  It  will  also  help  determine  if  tactics  need  to  be  adjusted  for  contracts  in 
different  phases  of  the  acquisition  lifecycle. 

Research  Question  #3 

Is  the  effect  of  the  FASA  and  the  Clinger-Cohen  Act  on  cost  overruns  sensitive  to 
the  branch  of  service  responsible  for  the  contract? 

Research  Hypothesis  #4 

Ha:  The  mean  final  cost  overrun  percentages  for  Army,  Navy  and  Air  Force 
contracts  completed  before  December  31,  1997  are  not  equal  to  the  mean  final 
cost  overrun  percentages  on  Army,  Navy  and  Air  Force  contracts  completed  after 
December  31,  1997. 
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This  research  question  was  motivated  by  Christensen,  Searle  and  Vickery’s 
research.  Since  they  found  that  cost  overruns  were  drastically  worse  for  the  Air  Force 
after  implementation  of  the  Packard  Commission’s  findings,  it  was  a  natural  curiosity  to 
see  how  the  respective  branches  of  service  have  been  performing  since  then.  This 
comparison  will  be  accomplished  by  using  the  Contract  ID  number  (CID)  to  identify 
which  branch  of  service  is  responsible  and  dividing  the  contracts  into  respective  samples. 
This  research  question  does  not  address  Marine  contracts  individually.  An  initial 
inspection  of  the  database  revealed  that  there  were  an  insufficient  number  of  Marine 
contracts  to  comprise  their  own  sample.  Since  the  Marines  are  part  of  the  Department  of 
the  Navy  (DoN),  these  contracts  were  rolled  into  the  Navy  sample  and  treated  as  Navy 
contracts.  For  testing  purposes,  this  hypothesis  will  be  divided  into  three  sub-hypotheses 
to  test  each  service  individually. 

Research  Question  #4 

Is  the  effect  of  the  FASA  and  Clinger-Cohen  Act  on  cost  overruns  sensitive  to  the 
type  of  contract? 

Research  Hypothesis  #5 

The  mean  final  overrun  percentage  on  fixed  price  contracts  completed  before  the 
implementation  of  the  FASA  and  Clinger-Cohen  Act  is  not  equal  to  the  mean 
final  overrun  percentage  on  fixed  price  contracts  completed  after  the 
implementation  of  the  FASA  and  Clinger-Cohen  Act. 

Research  Hypothesis  #6 

The  mean  final  overrun  percentage  on  cost  reimbursable  contracts  completed 
before  the  implementation  of  the  FASA  and  Clinger-Cohen  Act  is  not  equal  to  the 
mean  final  overrun  percentage  on  cost  reimbursable  contracts  completed  after  the 
implementation  of  the  FASA  and  Clinger-Cohen  Act. 
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Hypothesis  #5  and  Hypothesis  #6  will  determine  if  there  is  a  difference  in  the 
effect  the  FASA  and  Clinger-Cohen  Act  had  on  cost  overruns.  This  research  question 
was  motivated  by  the  fundamental  difference  in  contract  types.  In  a  fixed  price  contract, 
the  contractor  bears  the  financial  burden  of  a  cost  overrun.  Since  the  price  is  fixed,  a  cost 
overrun  must  be  absorbed  by  the  contractor’s  profit.  Conversely,  in  a  cost  reimbursable 
contract,  the  contractor  is  generally  reimbursed  the  total  cost  of  the  contract  plus  a  fixed 
fee.  Therefore,  a  cost  overrun  on  a  cost  reimbursable  contract  is  absorbed  by  the 
government.  This  basic  difference  of  how  cost  overruns  are  handled  between  the  two 
types  of  contracts  creates  a  natural  curiosity  that  this  research  can  satisfy. 

Hypothesis  Tests 

This  section  discusses  the  hypothesis  tests  used  to  compare  the  samples.  It  details 
how  the  tests  will  be  conducted,  presents  the  necessary  formulas,  and  also  provides  a 
discussion  of  any  key  assumptions  that  are  pertinent  to  draw  inferences  from  the  results 
of  the  tests.  A  flowchart,  which  graphically  depicts  the  logical  steps  taken  to  test  the 
hypotheses,  is  presented  at  the  end  of  this  section. 

The  two  sample  t-test  is  used  to  test  the  hypotheses  stated  above.  It  uses  the 
sample  means  of  the  final  overrun  percentage  and  tests  that  the  difference  between  the 
sample  means  is  zero.  If  the  test  does  show  that  the  difference  between  the  sample  means 
is  zero,  then  the  difference  between  the  population  means  can  be  inferred  to  be  zero  as 
well.  The  t-test  uses  a  standardized  variable  (t)  that  follows  a  t  distribution  with  v 
degrees  of  freedom.  The  t-test  statistic  and  degrees  of  freedom  are  calculated  using 
equations  3  and  4: 
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m  n 

-  +  - 

m  -  l  n  -  1  5  where 


x  =  sample  mean  from  the  first  sample, 
v  =sample  mean  from  the  second  sample, 

Si=  sample  variance  from  the  first  sample, 

S2=  sample  variance  from  the  second  sample, 
m  =  sample  size  of  the  first  sample, 
n  =  sample  size  of  the  second  sample. 

(Devore,  2000:  366) 

Since  the  research  questions  are  focused  on  determining  if  the  final  overrun 
percentages  are  equal  and  are  concerned  with  which  sample  is  greater,  a  two  tailed  test  is 
appropriate.  The  null  hypotheses  will  be  rejected  under  the  following  conditions: 

t  2  toJ2,v  01  t  "Si  ~ta/2,v 

The  two  sample  t-test  has  two  key  assumptions  that  should  be  satisfied  to  validate  its 
inferences.  The  first  assumption  is  that  the  observations  are  normally  distributed.  The 
nonnality  assumption  is  assessed  visually  and  quantitatively.  The  visual  assessment  is 
accomplished  by  constructing  a  nonnal  probability  plot  and  checking  its  linearity.  If  the 
observations  come  from  a  normal  distribution,  the  line  should  fall  close  to  a  straight  line 
(Devore,  2000:  190).  The  quantitative  assessment  for  normality  is  accomplished  using 
the  Shapiro-Wilk  test.  The  calculations  for  the  Shapiro-Wilk  test  are  performed  using  the 
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JMP  software  package,  version  5.0.  The  Shapiro-Wilk  test  yields  a  p-value  between  0 
and  1.  Normality  can  be  assumed  when  the  p-value  is  greater  than  0.05. 

Fortunately,  since  this  research  involves  using  averages  of  the  final  cost  overruns, 
it  is  not  absolutely  necessary  to  have  normally  distributed  data.  The  central  limit  theorem 
states  “if  [the  sample  size]  is  sufficiently  large,  the  [mean]  has  approximately  a  normal 
distribution”  (Devore,  2000:  235).  This  theorem  allows  the  two  sample  t-test  to  be  used 
in  the  absence  of  normality  and  still  have  confidence  in  the  resulting  inferences.  The 
theorem  is  somewhat  ambiguous  stating  that  the  sample  size  must  be  “sufficiently  large.” 
It  is  not  specific  because  the  number  of  required  observations  varies  from  sample  to 
sample.  Having  a  sample  size  greater  than  30  is  generally  accepted  as  a  sufficiently  large 
number  for  any  distribution  (Devore,  2000:  236).  For  this  research,  three  groups  of 
contracts  have  sample  sizes  less  than  30:  the  pre-1997  Army  contracts,  the  post-1997 
Army  contracts,  and  the  post- 1997  Air  Force  contracts.  As  shown  in  Table  6,  their 
respective  sample  sizes  are  19,  28,  and  27.  For  tests  involving  these  samples,  it  will  be 
important  for  the  results  of  the  t-test  to  be  corroborated  by  the  results  of  a  non-parametric 
test.  The  non-parametric  used  for  corroboration  is  the  Wilcoxon  Signed  Rank  test,  which 
does  not  require  the  assumption  of  normality. 

The  second  assumption  for  the  two  sample  t-test  is  that  the  observations  have 
constant  variance.  Constant  variance  will  be  tested  using  the  modified  Levene  test.  It  is 
a  robust  test  that  can  be  used  in  the  absence  of  nonnality  and  when  sample  sizes  are  not 
equal  (Neter  et  al,  1996:  766).  These  calculations  are  also  accomplished  using  JMP, 
version  5.0.  The  details  of  these  calculations  can  be  found  in  the  4th  edition  of  Applied 
Linear  Statistical  Models  by  Neter,  Kutner,  Nachtsheim,  and  Wasserman.  The  Levene 
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test  yields  a  p-value  which  needs  to  be  greater  than  0.05  to  assume  constant  variance. 
Conversely,  if  a  p-value  of  less  than  0.05  is  obtained,  it  cannot  be  assumed  that  the  data 
has  constant  variance.  For  the  samples  without  constant  variance,  it  is  important  that  the 
two  sample  t-test  be  corroborated  by  a  test  that  does  not  require  homogenous  variances. 
The  test  used  to  support  the  conclusions  drawn  with  the  two  sample  t-test  is  the  Welch 
ANOVA  test.  The  Welch  ANOVA  test  is  commonly  used  in  the  absence  of  non- 
homogenous  variances  to  test  whether  the  means  of  the  two  samples  are  equal.  In  this 
test,  the  means  are  weighted  by  the  reciprocal  of  the  sample  variances  of  the  group  means 
(JMP,  2002).  The  flowchart  in  Figure  3  shows  the  logic  involved  in  the  selection  of  the 
appropriate  hypothesis  tests. 


Figure  3.  Flowchart  for  Hypothesis  Test  Selection 
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Significance  Level  and  Errors 

The  level  of  significance  for  this  research  is  set  at  an  alpha  level  of  0.05.  In  other 
words,  this  is  the  smallest  p-value  at  which  the  null  hypotheses  are  not  rejected.  This 
level  alpha  is  used  for  the  hypothesis  tests  and  for  the  assumptions  of  normality  and 
constant  variance  tests.  For  the  hypothesis  tests,  using  an  alpha  level  of  0.05  allows  us  to 
be  confident  that  the  corresponding  means  fall  somewhere  between  the  upper  2.5%  and 
lower  2.5%  tails  of  the  t  distribution.  A  higher  p-value  means  we  are  even  more 
confident  that  the  mean  is  located  in  the  desired  area.  So  the  user  is  able  to  use  judgment 
in  the  interpretation  of  this  research,  the  associated  p-values  are  reported. 

When  drawing  conclusions  from  the  results  of  any  research  using  hypothesis 
testing  procedures,  it  is  inevitable  that  errors  may  occur.  It  is  possible  for  the  null 
hypothesis  to  be  rejected  when  it  should  not  be  rejected.  In  this  situation,  a  type  I  error  is 
said  to  have  occurred.  It  is  also  possible  to  fail  to  reject  the  null  hypothesis  when  it 
should  have  been  rejected.  This  error  is  known  as  a  Type  II  error.  The  only  way  to  have 
a  hypothesis  test  where  no  errors  can  occur  is  to  be  able  to  examine  the  entire  population 
(Devore,  2000:  3 14).  Since  many  of  the  contracts  in  the  DAES  database  were  excluded 
from  the  study  for  various  reasons  (e.g.  incomplete  data)  and  many  smaller  contracts  are 
not  included  in  the  database,  the  entire  population  of  DoD  contracts  is  not  available.  This 
research  uses  a  representative  sample;  however,  inferences  based  on  the  results  of  this 
study  should  be  interpreted  with  the  knowledge  that  these  two  errors  may  be  present. 
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IV.  Results 


Chapter  Overview 

This  chapter  presents  the  results  of  the  hypothesis  tests  along  with  the  results  of 
the  tests  for  the  assumptions  of  normality  and  constant  variance.  Each  hypothesis  is 
presented  individually  with  the  results  of  the  nonnality  and  constant  variance  tests  shown 
first  followed  by  the  associated  hypothesis  tests.  The  redundancy  in  this  chapter  is 
intended  to  allow  the  results  of  each  hypothesis  to  stand  alone.  This  method  of 
organization  gives  the  reader  some  flexibility  in  how  the  chapter  is  read.  The  results  and 
associated  statistics  for  each  hypothesis  are  summarized  at  the  end  of  the  chapter. 

Hypothesis  #1 

Normality  for  the  samples  corresponding  to  Hypothesis  #1  is  tested  both  visually 
and  quantitatively.  The  visual  test  is  accomplished  by  using  a  normal  probability  plot.  A 
perfectly  normal  distribution  is  depicted  in  a  normal  probability  plot  by  a  straight  line. 
Therefore,  an  approximate  straight  line  can  be  assumed  to  approach  a  nonnal  distribution. 
Figures  4  and  5  in  Appendix  A  show  the  normal  probability  plots  associated  with 
Hypothesis  #1.  Figure  4  is  constructed  using  the  Pre-1997  contracts,  and  Figure  5  is 
constructed  using  the  Post- 1997  contracts.  As  one  can  see,  the  plots  do  not  yield  straight 
lines,  so  it  is  likely  that  these  two  samples  are  not  normally  distributed.  The  quantitative 
test  for  nonnality,  the  Shapiro-Wilk  test,  yields  test  statistic  of  0.806  and  0.944 
respectively  and  p-values  of  less  than  0.0002.  Since  this  is  less  than  the  alpha  of  0.05, 
this  confirms  that  these  two  samples  do  not  come  from  populations  with  a  nonnal 
distribution.  See  Table  7  for  a  summary  of  the  tests  for  normality. 
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The  modified  Levene  test,  which  is  the  test  for  equal  variances,  yields  a  test 
statistic  of  26.9445  and  a  p-value  of  less  than  0.0001.  Since  the  p-value  is  less  than  the 
0.05  level  of  significance,  it  can  be  assumed  that  these  samples  do  not  have  constant 
variance.  See  Table  7  for  a  summary  of  the  tests  for  constant  variance. 

Even  though  the  assumption  of  normality  is  violated,  the  Central  Limit  Theorem 
stated  in  Chapter  3  allows  for  the  two  sample  t-test  to  be  used  in  the  absence  of 
nonnality.  Since  the  samples  do  not  have  constant  variance,  the  Welch  ANOVA  test  is 
also  performed  to  validate  the  results  of  the  t-test.  Using  the  t-test  to  test  that  the  mean  of 
the  first  sample  is  equal  to  the  mean  of  the  second  sample  yields  a  test  statistic  of  4.345 
and  a  p-value  of  less  than  0.0001.  The  Welch  ANOVA  test  supports  these  results  with  a 
test  statistic  of  18.2374  and  a  p-value  of  less  than  0.0001.  Therefore,  the  null  hypothesis 
is  rejected  and  one  can  assume  that  the  mean  final  overrun  percentage  of  the  pre-1997 
contracts  is  not  equal  to  the  mean  final  overrun  percentage  of  the  post- 1997  contracts. 
Since  the  mean  final  overrun  percentage  of  the  post- 1997  contracts  is  better  than  the 
mean  final  overrun  percentage  of  the  pre-1997  contracts,  one  can  assume  that  cost 
overruns  improved  after  the  treatment  date  of  December  31,  1997.  See  Table  7  for  a 
complete  summary  of  the  results  for  this  hypothesis. 

Hypothesis  #2 

Normality  for  the  samples  corresponding  to  Hypothesis  #2  is  tested  both  visually 
and  quantitatively.  The  visual  test  is  accomplished  by  using  a  normal  probability  plot. 
Figures  6  and  7  in  Appendix  A  show  the  normal  probability  plots  associated  with 
Hypothesis  #2.  Figure  6  is  constructed  using  the  Pre-1997  development  contracts,  and 
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Figure  7  is  constructed  using  the  Post- 1997  development  contracts.  As  one  can  see,  the 
plots  do  not  yield  straight  lines,  so  it  is  likely  that  these  two  samples  are  not  normally 
distributed.  The  quantitative  test  for  nonnality,  the  Shapiro-Wilk  test,  yields  test 
statistics  of  0.823  and  0.925  respectively  and  p-values  of  less  than  0.0007.  Since  the  p- 
value  is  less  than  the  alpha  of  0.05,  this  confirms  that  these  two  samples  to  not  come  from 
populations  with  a  normal  distribution.  See  Table  7  for  a  summary  of  the  tests  for 
nonnality. 

The  modified  Levene  test,  which  is  the  test  for  equal  variances,  yields  a  test 
statistic  of  17.1864  and  a  p-value  of  less  than  0.0001.  Since  the  p-value  is  less  than  the 
0.05  level  of  significance,  it  can  be  assumed  that  these  samples  do  not  have  constant 
variance.  See  Table  7  for  a  summary  of  the  tests  for  constant  variance. 

Even  though  the  assumption  of  normality  is  violated,  the  Central  Limit  Theorem 
stated  in  Chapter  3  allows  for  the  two  sample  t-test  to  be  used  in  the  absence  of 
nonnality.  Since  the  samples  do  not  have  constant  variance,  the  Welch  ANOVA  test  is 
also  performed  to  validate  the  results  of  the  t-test.  Using  the  two  sample  t-test  to  test  that 
the  mean  of  the  first  sample  is  equal  to  the  mean  of  the  second  sample  yields  a  test 
statistic  of  4.098  and  a  p-value  of  less  than  0.0001.  The  Welch  ANOVA  test  supports 
these  results  with  a  test  statistic  of  12.3937  and  a  p-value  of  0.0009.  Therefore,  the  null 
hypothesis  is  rejected  and  one  can  assume  that  the  mean  final  overrun  percentage  of  the 
pre-1997  development  contracts  is  not  equal  to  the  mean  final  overrun  percentage  of  the 
post- 1997  development  contracts.  Since  the  mean  final  overrun  percentage  of  the  post- 
1 997  development  contracts  is  better  than  the  mean  final  overrun  percentage  of  the  pre- 
1 997  development  contracts,  one  can  assume  that  cost  overruns  for  contracts  in  the 
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development  phase  of  the  acquisition  lifecycle  improved  after  the  treatment  date  of 
December  31,  1997.  See  Table  7  for  a  complete  summary  of  the  results  for  this 
hypothesis. 

Hypothesis  #3 

Normality  for  the  samples  corresponding  to  Hypothesis  #3  is  tested  both  visually 
and  quantitatively.  The  visual  test  is  accomplished  by  using  a  normal  probability  plot. 
Figures  8  and  9  in  Appendix  A  show  the  normal  probability  plots  associated  with 
Hypothesis  #3.  Figure  8  is  constructed  using  the  pre-1997  production  contracts,  and 
Figure  9  is  constructed  using  the  post- 1997  production  contracts.  As  one  can  see,  the 
plots  do  not  yield  straight  lines,  so  it  is  likely  that  these  two  samples  are  not  normally 
distributed.  The  quantitative  test  for  nonnality,  the  Shapiro-Wilk  test,  yields  test 
statistics  of  0.790  and  0.962  and  p-values  of  less  than  0.0001  and  0.1941,  respectively. 
Since  the  p-value  for  the  pre-1997  production  contracts  is  less  than  the  alpha  of  0.05,  this 
confirms  that  this  sample  does  not  come  from  a  population  with  a  normal  distribution. 
However,  based  on  the  p-value  associated  with  the  post- 1997  production  contracts,  this 
sample  appears  to  be  normally  distributed.  See  Table  7  for  a  summary  of  the  tests  for 
nonnality. 

The  modified  Levene  test,  which  is  the  test  for  equal  variances,  yields  a  test 
statistic  of  9.4562  and  a  p-value  of  0.0026.  Since  the  p-value  is  less  than  the  0.05  level  of 
significance,  it  can  be  assumed  that  these  samples  do  not  have  constant  variance.  See 
Table  7  for  a  summary  of  the  tests  for  constant  variance. 
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Even  though  the  assumption  of  normality  is  violated  with  one  of  these  samples, 
the  Central  Limit  Theorem  allows  for  the  two  sample  t-test  to  be  used  to  test  this 
hypothesis.  The  Central  Limit  Theorem  stated  in  Chapter  3  allows  for  the  two  sample  t- 
test  to  be  used  in  the  absence  of  nonnality.  Since  the  samples  do  not  have  constant 
variance,  the  Welch  ANOVA  test  is  also  performed  to  validate  the  results  of  the  t-test. 
Using  the  two  sample  t-test  to  test  that  the  mean  of  the  first  sample  is  equal  to  the  mean 
of  the  second  sample  yields  a  test  statistic  of  2.588  and  a  p-value  of  0.0109.  The  Welch 
ANOVA  test  supports  these  results  with  a  test  statistic  of  8.013 1  and  a  p-value  of  0.0056. 
Therefore,  the  null  hypothesis  is  rejected  and  one  can  assume  that  the  mean  final  overrun 
percentage  of  the  pre-1997  production  contracts  is  not  equal  to  the  mean  final  overrun 
percentage  of  the  post- 1997  production  contracts.  Since  the  mean  final  overrun 
percentage  of  the  post- 1997  production  contracts  is  better  than  the  mean  final  overrun 
percentage  of  the  pre-1997  production  contracts,  one  can  assume  that  cost  overruns  for 
contracts  in  the  production  phase  of  the  acquisition  lifecycle  improved  after  the  treatment 
date  of  December  31,  1997.  See  Table  7  for  a  complete  summary  of  the  results  for  this 
hypothesis. 

Hypothesis  #4 

The  final  hypothesis  is  divided  in  three  sub-hypotheses  dealing  with  Army,  Navy, 
and  Air  Lorce  contracts  separately.  Each  sub-hypothesis  has  its  own  tests  for  nonnality 
and  constant  variance  along  with  an  associated  two  sample  t-test  to  assess  each  sub¬ 
hypothesis. 
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Army  Contracts 

Normality  for  the  samples  corresponding  to  sub-hypothesis  #1  of  Hypothesis  #4  is 
tested  both  visually  and  quantitatively.  The  visual  test  is  accomplished  by  using  a  nonnal 
probability  plot.  Figures  10  and  1 1  in  Appendix  A  show  the  normal  probability  plots 
associated  with  sub-hypothesis  #1  of  Hypothesis  #4.  Figure  10  is  constructed  using  the 
pre-1997  Anny  contracts,  and  Figure  1 1  is  constructed  using  the  post- 1997  Army 
contracts.  As  one  can  see,  the  plots  do  not  yield  straight  lines,  so  it  is  likely  that  these 
two  samples  are  not  normally  distributed.  The  quantitative  test  for  nonnality,  the 
Shapiro-Wilk  test,  yields  test  statistics  of  0.707  and  0.908  and  p-values  of  less  than 
0.0001  and  0.0189,  respectively.  Since  the  p-values  for  these  two  samples  are  less  than 
the  alpha  of  0.05,  this  confirms  that  these  samples  do  not  come  from  a  population  with  a 
nonnal  distribution.  See  Table  7  for  a  summary  of  the  tests  for  normality. 

The  modified  Levene  test,  which  is  the  test  for  equal  variances,  yields  a  test 
statistic  of  3.6796  and  a  p-value  of  0.0614.  Since  the  p-value  is  greater  than  the  0.05 
level  of  significance,  it  can  be  assumed  that  these  samples  do  have  constant  variance. 

See  Table  7  for  a  summary  of  the  tests  for  constant  variance. 

Since  the  assumption  of  normality  is  violated  with  these  two  samples  and  the 
sample  sizes  are  too  small  to  invoke  the  Central  Limit  Theorem,  the  two  sample  t-test 
must  be  corroborated  by  the  Wilcoxon  Signed  Rank  test.  Using  the  two  sample  t-test  to 
test  that  the  mean  of  the  first  sample  is  equal  to  the  mean  of  the  second  sample  yields  a 
test  statistic  of  2.591  and  a  p-value  of  0.0128.  The  Wilcoxon  Signed  Rank  test  supports 
these  results  with  a  test  statistic  of  -2.265  and  a  p-value  of  0.0235.  Therefore,  the  null 
hypothesis  is  rejected  and  one  can  assume  that  the  mean  final  overrun  percentage  of  the 
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pre-1997  Army  contracts  is  not  equal  to  the  mean  final  overrun  percentage  of  the  post- 
1997  Army  contracts.  Since  the  mean  final  overrun  percentage  of  the  post- 1997  Army 
contracts  is  better  than  the  mean  final  overrun  percentage  of  the  pre-1997  Army 
contracts,  one  can  assume  that  cost  overruns  on  Army  contracts  improved  after  the 
treatment  date  of  December  31,  1997.  See  Table  7  for  a  complete  summary  of  the  results 
for  this  hypothesis. 

Navy  Contracts 

Normality  for  the  samples  corresponding  to  sub-hypothesis  #2  of  Hypothesis  #4  is 
tested  both  visually  and  quantitatively.  The  visual  test  is  accomplished  by  using  a  nonnal 
probability  plot.  Figures  12  and  13  in  Appendix  A  show  the  normal  probability  plots 
associated  with  sub-hypothesis  #2  of  Hypothesis  #4.  Figure  12  is  constructed  using  the 
pre-1997  Navy  contracts,  and  Figure  13  is  constructed  using  the  post- 1997  Navy 
contracts.  As  one  can  see,  the  plots  do  not  yield  straight  lines,  so  it  is  likely  that  these 
two  samples  are  not  normally  distributed.  The  quantitative  test  for  nonnality,  the 
Shapiro-Wilk  test,  yields  test  statistics  of  0.857  and  0.95 1  and  p-values  of  less  than 
0.0001  and  0.0384,  respectively.  Since  the  p-values  for  these  two  samples  are  less  than 
the  alpha  of  0.05,  this  confirms  that  these  samples  do  not  come  from  a  population  with  a 
nonnal  distribution.  See  Table  7  for  a  summary  of  the  tests  for  normality. 

The  modified  Levene  test,  which  is  the  test  for  equal  variances,  yields  a  test 
statistic  of  2 1 .997  and  a  p-value  of  less  than  0.000 1 .  Since  the  p-value  is  less  than  the 
0.05  level  of  significance,  it  can  be  assumed  that  these  samples  do  not  have  constant 
variance.  See  Table  7  for  a  summary  of  the  tests  for  constant  variance. 
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Even  though  the  assumption  of  normality  is  violated  with  these  two  samples,  the 
Central  Limit  Theorem  stated  in  Chapter  Three  allows  for  the  two  sample  t-test  to  be 
used  to  test  this  hypothesis.  Since  the  samples  do  not  have  constant  variance,  the  Welch 
ANOVA  test  is  also  performed  to  validate  the  results  of  the  t-test.  Using  the  two  sample 
t-test  to  test  that  the  mean  of  the  first  sample  is  equal  to  the  mean  of  the  second  sample 
yields  a  test  statistic  of  3.578  and  a  p-value  of  0.0005.  The  Welch  ANOVA  test  supports 
these  results  with  a  test  statistic  of  1 1.0182  and  a  p-value  of  0.0016.  Therefore,  the  null 
hypothesis  is  rejected  and  one  can  assume  that  the  mean  final  overrun  percentage  of  the 
pre-1997  Navy  contracts  is  not  equal  to  the  mean  final  overrun  percentage  of  the  post- 
1997  Navy  contracts.  Since  the  mean  final  overrun  percentage  of  the  post- 1997  Navy 
contracts  is  better  than  the  mean  final  overrun  percentage  of  the  pre-1997  Navy  contracts, 
one  can  assume  that  cost  overruns  on  Navy  contracts  improved  after  the  treatment  date  of 
December  31,  1997.  See  Table  7  for  a  complete  summary  of  the  results  for  this 
hypothesis. 

Air  Force  Contracts 

Normality  for  the  samples  corresponding  to  sub-hypothesis  #3  of  Hypothesis  #4  is 
tested  both  visually  and  quantitatively.  The  visual  test  is  accomplished  by  using  a  nonnal 
probability  plot.  Figures  14  and  15  in  Appendix  A  show  the  normal  probability  plots 
associated  with  sub-hypothesis  #3  of  Hypothesis  #4.  Figure  14  is  constructed  using  the 
pre-1997  Air  Force  contracts,  and  Figure  15  is  constructed  using  the  post- 1997  Air  Force 
contracts.  As  one  can  see,  the  plots  do  not  yield  straight  lines,  so  it  is  likely  that  these 
two  samples  are  not  normally  distributed.  The  quantitative  test  for  nonnality,  the 
Shapiro-Wilk  test,  yields  test  statistics  of  0.750  and  0.936  and  p-values  of  less  than 
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0.0001  and  0. 1043,  respectively.  Since  the  p-value  for  the  pre-1997  Air  Force  contracts 
is  less  than  the  alpha  of  0.05,  this  confirms  that  this  sample  does  not  come  from  a 
population  with  a  normal  distribution.  Based  on  the  p-value  associated  with  the  post- 
1997  Air  Force  contracts,  this  sample  appears  to  be  normally  distributed.  Like  the  post- 
1997  production  contracts,  this  sample  also  shows  why  a  visual  check  is  not  completely 
adequate  when  testing  for  normality.  See  Table  7  for  a  summary  of  the  tests  for 
nonnality. 

The  modified  Levene  test,  which  is  the  test  for  equal  variances,  yields  a  test 
statistic  of  4.4692  and  a  p-value  of  less  than  0.0383.  Since  the  p-value  is  less  than  the 
0.05  level  of  significance,  it  can  be  assumed  that  these  samples  do  not  have  constant 
variance.  See  Table  7  for  a  summary  of  the  tests  for  constant  variance. 

Even  though  the  assumption  of  normality  is  violated  with  one  of  these  samples, 
the  Central  Limit  Theorem  stated  in  Chapter  Three  allows  for  the  two  sample  t-test  to  be 
used  to  test  this  hypothesis.  Since  the  samples  do  not  have  constant  variance,  the  Welch 
ANOVA  test  is  also  performed  to  validate  the  results  of  the  t-test.  Using  the  two  sample 
t-test  to  test  that  the  mean  of  the  first  sample  is  equal  to  the  mean  of  the  second  sample 
yields  a  test  statistic  of  1.580  and  a  p-value  of  0.1 188.  The  Welch  ANOVA  test  supports 
these  results  with  a  test  statistic  of  3.2134  and  a  p-value  of  0.0779.  Therefore,  the  null 
hypothesis  is  not  rejected  and  one  can  assume  that  the  mean  final  overrun  percentage  of 
the  pre-1997  Air  Force  contracts  is  equal  to  the  mean  final  overrun  percentage  of  the 
post- 1997  Air  Force  contracts.  Since  the  mean  final  overrun  percentage  of  the  post- 1997 
Air  Force  contracts  is  statistically  the  same  as  the  mean  final  overrun  percentage  of  the 
pre-1997  Air  Force  contracts,  one  can  assume  that  cost  overruns  on  Air  Force  contracts 
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experienced  no  change  after  the  treatment  date  of  December  31,  1997.  See  Table  7  for  a 
complete  summary  of  the  results  for  this  hypothesis. 

Hypothesis  #5 

Normality  for  the  samples  corresponding  to  Hypothesis  #5  is  tested  both  visually 
and  quantitatively.  The  visual  test  is  accomplished  by  using  a  normal  probability  plot. 
Figures  16  and  17  in  Appendix  A  show  the  normal  probability  plots  associated  with 
Hypothesis  #5.  Figure  16  is  constructed  using  the  pre-1997  fixed  price  contracts,  and 
Figure  17  is  constructed  using  the  post- 1997  fixed  price  contracts.  As  one  can  see,  the 
plots  do  not  yield  straight  lines,  so  it  is  likely  that  these  two  samples  are  not  normally 
distributed.  The  quantitative  test  for  nonnality,  the  Shapiro-Wilk  test,  yields  test 
statistics  of  0.913  and  0.934  and  p-values  of  0.0060  and  0.0074,  respectively.  Since  the 
p-values  for  these  two  samples  are  less  than  the  alpha  of  0.05,  this  confirms  that  these 
samples  do  not  come  from  normally  distributed  populations.  See  Table  7  for  a  summary 
of  the  tests  for  nonnality. 

The  modified  Levene  test,  which  is  the  test  for  equal  variances,  yields  a  test 
statistic  of  15.872  and  a  p-value  of  0.0001.  Since  the  p-value  is  less  than  the  0.05  level  of 
significance,  it  can  be  assumed  that  these  samples  do  not  have  constant  variance.  See 
Table  7  for  a  summary  of  the  tests  for  constant  variance. 

Even  though  the  assumption  of  normality  is  violated  with  these  two  samples,  the 
Central  Limit  Theorem  allows  for  the  two  sample  t-test  to  be  used  to  test  this  hypothesis. 
Since  the  samples  do  not  have  constant  variance,  the  Welch  ANOVA  test  is  also 
performed  to  validate  the  results  of  the  t-test.  Using  the  two  sample  t-test  to  test  that  the 
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mean  of  the  first  sample  is  equal  to  the  mean  of  the  second  sample  yields  a  test  statistic  of 
2.364  and  a  p-value  of  0.0202.  The  Welch  ANOVA  test  supports  these  results  with  a  test 
statistic  of  4.8804  and  a  p-value  of  0.03 1 .  Therefore,  the  null  hypothesis  is  rejected  and 
one  can  assume  that  the  mean  final  overrun  percentage  of  the  pre-1997  fixed  price 
contracts  is  not  equal  to  the  mean  final  overrun  percentage  of  the  post- 1997  fixed  price 
contracts.  Since  the  mean  final  overrun  percentage  of  the  post- 1997  fixed  price  contracts 
is  better  than  the  mean  final  overrun  percentage  of  the  pre-1997  fixed  price  contracts,  one 
can  assume  that  cost  overruns  for  fixed  price  contracts  improved  after  the  treatment  date 
of  December  31,  1997.  See  Table  7  for  a  complete  summary  of  the  results  for  this 
hypothesis. 

Hypothesis  #6 

Normality  for  the  samples  corresponding  to  Hypothesis  #6  is  tested  both  visually 
and  quantitatively.  The  visual  test  is  accomplished  by  using  a  normal  probability  plot. 
Figures  18  and  19  in  Appendix  A  show  the  normal  probability  plots  associated  with 
Hypothesis  #6.  Figure  18  is  constructed  using  the  pre-1997  cost  reimbursable  contracts, 
and  Figure  19  is  constructed  using  the  post- 1997  cost  reimbursable  contracts.  As  one  can 
see,  the  plots  do  not  yield  straight  lines,  so  it  is  likely  that  these  two  samples  are  not 
normally  distributed.  The  quantitative  test  for  nonnality,  the  Shapiro-Wilk  test,  yields 
test  statistics  of  0.756  and  0.941  and  p-values  of  less  than  0.0001  and  0.0105, 
respectively.  Since  the  p-values  for  these  two  samples  are  less  than  the  alpha  of  0.05,  this 
confirms  that  these  samples  do  not  come  from  normally  distributed  populations.  See 
Table  7  for  a  summary  of  the  tests  for  normality. 
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The  modified  Levene  test,  which  is  the  test  for  equal  variances,  yields  a  test 
statistic  of  15.5955  and  a  p-value  of  0.0001.  Since  the  p-value  is  less  than  the  0.05  level 
of  significance,  it  can  be  assumed  that  these  samples  do  not  have  constant  variance.  See 
Table  7  for  a  summary  of  the  tests  for  constant  variance. 

Even  though  the  assumption  of  normality  is  violated  with  these  two  samples,  the 
Central  Limit  Theorem  allows  for  the  two  sample  t-test  to  be  used  to  test  this  hypothesis. 
Since  the  samples  do  not  have  constant  variance,  the  Welch  ANOVA  test  is  also 
performed  to  validate  the  results  of  the  t-test.  Using  the  two  sample  t-test  to  test  that  the 
mean  of  the  first  sample  is  equal  to  the  mean  of  the  second  sample  yields  a  test  statistic  of 
3.582  and  a  p-value  of  0.0005.  The  Welch  ANOVA  test  supports  these  results  with  a  test 
statistic  of  14.016  and  a  p-value  of  0.0003.  Therefore,  the  null  hypothesis  is  rejected  and 
one  can  assume  that  the  mean  final  overrun  percentage  of  the  pre-1997  cost  reimbursable 
contracts  is  not  equal  to  the  mean  final  overrun  percentage  of  the  post- 1997  cost 
reimbursable  contracts.  Since  the  mean  final  overrun  percentage  of  the  post- 1997  cost 
reimbursable  contracts  is  better  than  the  mean  final  overrun  percentage  of  the  pre-1997 
cost  reimbursable  contracts,  one  can  assume  that  cost  overruns  for  cost  reimbursable 
contracts  improved  after  the  treatment  date  of  December  31,  1997.  See  Table  7  for  a 
complete  summary  of  the  results  for  this  hypothesis. 
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Table  7.  Statistical  Analysis  Summary 


Hypothesis  1  Total  Population  of  Contracts 


Sample  1 : 

Total  Pre-1997  Contracts:  107  values  ranging  from  -134.82  to  22.04 

Sample  2: 

Total  Post-1997  Contracts:  113  values  ranging  from  -48.87  to  25.96 

Sample  1 

Sample  2 

Shapiro- Wilk 

0.806 

0.944 

p-value 

0 

0.0002 

Mean  FO% 

-11.79 

0.85 

Standard  Deviation 

27.83 

13.17 

Modified  Levene 

26.945 

p-value 

0.0001 

Test  Statistic 

p-value 

Two  Sample  t-test 

4.345 

0.0001 

Welch  ANOVA 

4.271 

0.0001 

Wilcoxon  Signed  Rank 

-3.393 

0.0007 

Hypothesis  2  Development  Contracts 


Sample  1 : 

Pre-1997  Development  Contracts:  41  values  ranging  from  -134.82  to  22.04 

Sample  2: 

Post-1997  Development  Contracts:  64  values  ranging  from  -48.87  to  22.68 

Sample  1 

Sample  2 

Shapiro- Wilk 

0.823 

0.925 

p-value 

0.0001 

0.0007 

Mean  FO% 

-18.86 

-0.29 

Standard  Deviation 

31.89 

13.89 

Modified  Levene 

17.186 

p-value 

0.0001 

Test  Statistic 

p-value 

Two  Sample  t-test 

4.098 

0.0001 

Welch  ANOVA 

3.52 

0.0009 

Wilcoxon  Signed  Rank 

-3.445 

0.0006 

49 


Hypothesis  3  Production  Contracts 


Sample  1 : 

Pre-1997  Production  Contracts:  66  values  ranging  from  -96.88  to  20.81 

Sample  2: 

Post-1997  Production  Contracts:  49  values  ranging  from  -32.71  to  25.96 

Sample  1 

Sample  2 

Shapiro-Wilk 

0.79 

0.962 

p-value 

0.0001 

0.1941 

Mean  FO% 

-7.39 

2.36 

Standard  Deviation 

24.21 

12.13 

Modified  Levene 

9.4562 

p-value 

0.0026 

Test  Statistic 

p-value 

Two  Sample  t-test 

2.588 

0.0109 

Welch  ANOVA 

2.831 

0.0056 

Wilcoxon  Signed  Rank 

1.971 

0.0487 

Hypothesis  4.1  Army  Contracts 


Sample  1 : 

Pre-1997  Army  Contracts:  19  values  ranging  from  -134.82  to  7.76 

Sample  2: 

Post-1997  Army  Contracts:  28  values  ranging  from  -48.87  to  23.29 

Sample  1 

Sample  2 

Shapiro-Wilk 

0.707 

0.908 

p-value 

0.0001 

0.0189 

Mean  FO% 

-18.42 

0.86 

Standard  Deviation 

33.47 

17.25 

Modified  Levene 

3.679 

p-value 

0.0614 

Test  Statistic 

p-value 

Two  Sample  t-test 

2.591 

0.0128 

Welch  ANOVA 

2.311 

0.0295 

Wilcoxon  Signed  Rank 

-2.265 

0.0235 
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Hypothesis  4.2  Navy  Contracts 


Sample  1 : 

Pre-1997  Navy  Contracts:  47  values  ranging  from  -94.84  to  20.63 

Sample  2: 

Post-1997  Navy  Contracts:  58  values  ranging  from  -32.71  to  25.96 

Sample  1 

Sample  2 

Shapiro-Wilk 

0.857 

0.951 

p-value 

0.0001 

0.0384 

Mean  FO% 

-13.33 

0.52 

Standard  Deviation 

26.82 

11.07 

Modified  Levene 

21.997 

p-value 

0.0001 

Test  Statistic 

p-value 

Two  Sample  t-test 

3.578 

0.0005 

Welch  ANOVA 

3.319 

0.0016 

Wilcoxon  Signed  Rank 

-2.678 

0.0074 

Hypothesis  4.3  Air  Force  Contracts 


Sample  1 : 

Pre-1997  Air  Force  Contracts:  41  values  ranging  from  -97.36  to  22.04 

Sample  2: 

Post-1997  Air  Force  Contracts:  27  values  ranging  from  -31 .91  to  24.52 

Sample  1 

Sample  2 

Shapiro-Wilk 

0.75 

0.936 

p-value 

0.0001 

0.1043 

Mean  FO% 

-6.95 

1.58 

Standard  Deviation 

25.96 

12.97 

Modified  Levene 

4.469 

p-value 

0.0383 

Test  Statistic 

p-value 

Two  Sample  t-test 

1.58 

0.1188 

Welch  ANOVA 

1.793 

0.0779 

Wilcoxon  Signed  Rank 

1.128 

0.2593 
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Hypothesis  5  Fixed  Price  Contracts 


Sample  1 : 

Pre-1997  Fixed  Price  Contracts:  39  values  ranging  from  -46.99  to  22.04 

Sample  2: 

Post-1997  Fixed  Price  Contracts:  54  values  ranging  from  -37.17  to  22.68 

Sample  1 

Sample  2 

Shapiro-Wilk 

0.914 

0.934 

p-value 

0.0060 

0.0074 

Mean  FO% 

-7.85 

0.30 

Standard  Deviation 

20.23 

13.00 

Modified  Levene 

15.872 

p-value 

0.0001 

Test 

Statistic 

p-value 

Two  Sample  t-test 

2.364 

0.0202 

Welch  ANOVA 

2.209 

0.0310 

Wilcoxon  Signed  Rank 

-1.491 

0.1360 

Hypothesis  6  Cost  Reimbursable 
Contracts 


Sample  1 : 

Pre-1997  Cost  Reimbursable  Contracts:  67  values  ranging  from  -134.82  to  20.63 

Sample  2: 

Post-1997  Cost  Reimbursable  Contracts:  59  values  ranging  from  -48.87  to  25.96 

Sample  1 

Sample  2 

Shapiro-Wilk 

0.756 

0.941 

p-value 

0.0001 

0.0105 

Mean  FO% 

-14.40 

1.37 

Standard  Deviation 

31.39 

13.41 

Modified  Levene 

15.596 

p-value 

0.0001 

Test  Statistic 

p-value 

Two  Sample  t-test 

3.582 

0.0005 

Welch  ANOVA 

3.744 

0.0003 

Wilcoxon  Signed  Rank 

3.036 

0.0024 
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V.  Conclusions 


Chapter  Overview 

This  chapter  discusses  the  findings  of  this  research  along  with  the  potential  impact 
of  those  findings  to  the  DoD.  It  begins  with  a  restatement  of  the  research  questions 
followed  by  a  discussion  relating  the  results  of  the  hypothesis  tests  to  the  research 
questions.  Then,  some  limitations  of  the  research  are  presented  with  a  discussion  of  how 
the  limitations  affect  the  interpretation  of  the  results.  The  chapter  concludes  with  some 
suggestions  for  future  research  that  would  complement  this  study. 

Research  Questions 

Recall  that  the  purpose  of  this  study  was  to  determine  what  effect  the  Federal 
Acquisition  Streamlining  Act  (FASA)  and  the  Clinger-Cohen  Act  had  on  cost  overruns  in 
DoD  contracts.  Specifically,  did  cost  overruns  improve,  worsen  or  stay  the  same  after  the 
implementation  of  the  FASA  and  the  Clinger-Cohen  Act?  To  make  this  determination, 
the  following  research  questions  were  posed: 

1 .  Is  the  final  cost  overrun  percentage  on  contracts  completed  after  the 
implementation  of  the  FASA  and  Clinger-Cohen  Act  equal  to  the  final  cost 
overrun  percentage  on  contracts  completed  before  the  implementation  of  the 
FASA  and  Clinger-Cohen  Act? 

2.  Is  the  final  cost  overrun  percentage  sensitive  to  the  phase  of  the  acquisition  life 
cycle  the  program  is  in? 

3.  Is  the  final  cost  overrun  percentage  sensitive  to  the  branch  of  service  responsible 
for  the  program? 
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4.  Is  the  final  overrun  percentage  sensitive  to  the  type  of  contract? 

The  results  of  the  hypothesis  tests  suggest  that  the  FASA  and  Clinger-Cohen  Act  have 
had  a  positive  impact  on  cost  overruns  within  DoD  programs.  So,  to  answer  the  first 
research  question,  the  final  cost  overrun  percentage  has  improved  for  DoD  contracts  after 
the  implementation  of  the  FASA  and  the  Clinger-Cohen  Act.  This  suggests  that 
Congress  and  DoD  leadership  have  made  some  effective  strides  with  current  acquisition 
reform  initiatives. 

The  second  research  question  is  aimed  at  detennining  if  the  improvements  in  cost 
overruns  are  sensitive  to  the  phase  of  the  acquisition  lifecycle.  The  results  indicate  that 
cost  overruns  have  improved  in  both  development  and  production  contracts  since  the 
implementation  of  the  FASA  and  the  Clinger-Cohen  Act.  Thus,  the  improvements  are 
not  sensitive  to  the  phase  of  the  acquisition  lifecycle.  This  effect  of  the  FASA  and  the 
Clinger-Cohen  Act  is  contrary  to  how  the  Packard  Commission’s  recommendations 
affected  cost  overruns.  The  Packard  Commission  recommendations  contributed  to  an 
increase  in  cost  overruns  for  development  contracts  while  cost  overruns  in  production 
contracts  experienced  no  change  (Searle,  1997:  72). 

The  third  research  question  addressed  whether  the  improvements  in  cost  overruns 
are  sensitive  to  the  branch  of  service  responsible  for  the  contract.  The  results  show  that 
cost  overruns  improved  in  both  Army  and  Navy  contracts.  Air  Force  contracts,  however, 
did  not  change  enough  to  conclude  with  confidence  that  the  populations  are  statistically 
different.  Therefore,  the  FASA  and  the  Clinger-Cohen  Act  affected  no  change  on  Air 
Force  contracts. 
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The  fourth  research  question  asks  whether  or  not  the  improvement  in  cost 
overruns  is  sensitive  to  the  type  of  contract.  The  results  show  that  cost  overruns  for  fixed 
price  and  cost  reimbursable  contracts  improved  after  the  implementation  of  the  FASA 
and  Clinger-Cohen  Act.  Therefore,  the  improvement  in  cost  overruns  is  not  sensitive  to 
the  type  of  contract,  which  implies  that  policy  changes  specific  to  contract  type  is  not 
necessary. 

Limitations 

Before  the  conclusions  are  used  for  any  decision  making,  there  are  four  important 
limitations  that  must  be  considered.  First,  the  treatment  date  of  December  31,  1997  may 
not  reflect  the  full  implementation  of  the  FASA  and  the  Clinger-Cohen  Act.  This  date 
was  selected  based  on  past  research  and  the  expert  opinion  of  acquisition  professionals. 
Since  the  selection  process  was  not  scientific,  a  date  other  than  the  one  selected  could 
better  represent  the  implementation  of  the  two  acts.  Further  analysis  would  determine  if 
the  results  are  sensitive  to  slight  changes  in  the  treatment  date. 

The  second  limitation  involves  the  contracts  contained  in  the  DAES  database. 
There  are  times  when  contracts  are  cancelled  for  various  reasons,  poor  cost  performance 
not  withstanding.  Using  contracts  that  are  75%  complete  excludes  those  contracts  that 
were  cancelled  before  the  75%  completion  point.  Including  those  contracts  could  yield 
different  results.  More  information  is  necessary  to  establish  which  contracts  were 
cancelled  and  to  ascertain  whether  this  limitation  had  a  pivotal  effect  on  the  results. 

Linking  the  acquisition  refonn  initiatives  to  the  cost  perfonnance  of  DoD 
contracts  is  the  third  limitation  of  this  study.  The  decrease  in  cost  overruns  noted  in  the 
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results  of  this  study  may  in  fact  be  coincidental.  The  assumption  is  that  the  FASA  and 
Clinger-Cohen  Act  caused  the  decrease  in  cost  overruns.  However,  without  a  more 
rigorous  statistical  analysis,  this  cause  and  effect  can  only  be  assumed. 

The  final  limitation  of  this  study  is  the  inability  of  it  to  capture  cost  savings  in 
DoD  acquisitions.  Reducing  the  cost  at  which  the  government  is  able  to  procure  items  is 
considered  a  positive  impact  of  acquisition  reform  initiatives.  Since  many  of  these  cost 
savings  occur  prior  to  the  awarding  of  the  contract,  the  savings  are  not  immediately 
discernible.  To  quantify  this  impact,  one  would  have  to  look  at  contracts  on  similar  items 
before  and  after  the  treatment  date  to  see  if  the  cost  per  item  decreased.  This  limitation  is 
very  important  in  how  the  results  are  interpreted.  For  example,  if  the  Air  Force  had 
plentiful  cost  savings  that  were  not  captured  in  the  final  overrun  percentages,  then  it  is 
possible  that  the  FASA  and  the  Clinger-Cohen  Act  led  to  those  cost  savings.  Therefore, 
the  impact  of  the  FASA  and  Clinger-Cohen  Act  may  be  much  greater  than  the  change  in 
the  final  overrun  percentage  indicates. 

Recommended  Future  Research 

Since  this  research  indicates  that  the  FASA  and  the  Clinger-Cohen  Act 
contributed  to  the  reduction  of  cost  overruns  on  DoD  acquisitions,  any  future  research 
that  strengthens  those  assertions  would  greatly  benefit  this  study.  This  section  outlines 
three  areas  of  recommended  research  that  could  increase  the  confidence  in  the  results  of 
this  analysis. 

First,  there  is  an  indication  that  a  different  methodology  yields  different  results. 
Similar  research  was  accomplished  using  an  alternate  definition  of  cost  variance.  Using 
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the  same  treatment  date,  Captain  Mark  Holbrook  used  the  following  equation  to  calculate 
the  final  cost  overrun  percentage: 

ACWP-BCWP 

BCWP 

His  conclusion  was  that  current  acquisition  reform  had  no  effect  on  the  final  cost 
overrun  percentage  of  DoD  contracts.  Since  this  conclusion  is  contrary  to  the  results 
shown  in  this  research,  further  analysis  would  benefit  both  Capt  Holbrook’s  research  and 
this  research.  A  comparison  of  both  methods  used  to  calculate  cost  overruns  would 
determine  which  method  is  more  appropriate.  The  results  of  a  comparative  analysis 
would  profoundly  impact  how  these  two  efforts  are  interpreted. 

The  second  area  of  recommended  research  is  to  subject  some  of  the  constraints  of 
this  research  to  a  sensitivity  analysis  to  detennine  if  slight  shifts  in  the  methodology  yield 
different  results.  Two  constraints  that  are  suited  for  a  sensitivity  analysis  are  the  dates 
used  to  capture  the  impact  of  the  FASA  and  the  Clinger-Cohen  Act  and  the  percentage 
completion  of  the  contracts.  A  combination  of  dates  with  varying  pivot  dates  should  be 
analyzed  and  compared  to  see  how  sensitive  the  results  are  to  the  selected  dates. 

Graphing  the  changes  in  the  final  cost  overrun  percentages  would  give  further  insight  into 
how  long  acquisition  reform  legislation  takes  to  impact  the  DoD  procurement  process. 
The  results  of  this  analysis  hinge  on  the  75%  completion  point  being  accurate.  As  stated 
previously,  this  completion  point  was  used  because  past  research  has  shown  that  the 
majority  of  costs  are  incurred  at  this  point  and  the  contract  can  be  considered  complete  in 
tenns  of  cost  (Searle,  1997:  41).  Testing  contract  completion  percentages  greater  than 
75%  would  show  whether  cost  overruns  worsen,  improve  or  stay  the  same  after  the  75% 
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completion  point.  Drastic  changes  in  the  cost  overrun  percentage  after  the  75% 
completion  point  could  contradict  the  results  of  this  effort. 

The  third  area  of  recommended  research  would  give  deeper  insight  into  the 
Clinger-Cohen  Act.  Sandra  Meckley  believes  that  much  of  the  impact  of  the  Clinger- 
Cohen  Act  is  just  now  being  felt  (Meckley,  2002).  This  could  be  due  to  the  increased 
emphasis  on  Infonnation  Technology  (IT)  resources  in  recent  years.  To  determine  the 
validity  of  Ms.  Meckley’s  assumption,  one  would  need  to  isolate  the  contracts  dealing 
with  IT  resources  and  perform  a  hypothesis  test  with  only  those  contracts  using  a  more 
recent  treatment  date. 
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Observation  Observation 


Appendix  A 


Normal  Probability  Plots 


Figure  4.  Normal  Probability  Plot  (Pre-1997  Contracts) 


Figure  5.  Normal  Probability  Plot  (Post-1997  Contracts) 
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Figure  6.  Normal  Probability  Plot  (Pre-1997  Development  Contracts) 


Figure  7.  Normal  Probability  Plot  (Post-1997  Development  Contracts) 
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Figure  8.  Normal  Probability  Plot  (Pre-1997  Production  Contracts) 


Figure  9.  Normal  Probability  Plot  (Post-1997  Production  Contracts) 
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Figure  10.  Normal  Probability  Plot  (Pre-1997  Army  Contracts) 


Figure  11.  Normal  Probability  Plot  (Post-1997  Army  Contracts) 
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Figure  12.  Normal  Probability  Plot  (Pre-1997  Navy  Contracts) 


Figure  13.  Normal  Probability  Plot  (Post-1997  Navy  Contracts) 
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Figure  14.  Normal  Probability  Plot  (Pre-1997  Air  Force  Contracts) 


Figure  15.  Normal  Probability  Plot  (Post-1997  Air  Force  Contracts) 
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Figure  16.  Normal  Probability  Plot  (Pre-1997  Fixed  Price  Contracts) 


Figure  17.  Normal  Probability  Plot  (Post-1997  Fixed  Price  Contracts) 
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Figure  18.  Normal  Probability  Plot  (Pre-1997  Cost  Reimbursable  Contracts) 


Figure  19.  Normal  Probability  Plot  (Post-1997  Cost  Reimbursable  Contracts) 
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Appendix  B 


Data  Table  For  Contracts  Used  In  This  Study 


SUBMITDATE 

SERVICE 

PPHASE 

CTYPE 

BAC 

ACWP 

FO 

FO% 

%COMP 

1/25/1994 

Navy 

Production 

CR 

108.8 

109.4 

-0.6 

-0.55147 

99.26471 

1/25/1994 

Navy 

R&D 

CR 

40.7 

79.3 

-38.6 

-94.8403 

199.7543 

3/25/1994 

Navy 

Production 

CR 

83.8 

139 

-55.2 

-65.8711 

99.40334 

4/25/1994 

Navy 

Production 

CR 

1019.4 

1004 

15.4 

1.510693 

98.22445 

5/25/1994 

Army 

R&D 

FP 

14.7 

21.3 

-6.6 

-44.898 

125.1701 

5/25/1994 

Air  Force 

R&D 

CR 

873.9 

1088.5 

-214.6 

-24.5566 

124.4536 

7/25/1994 

Navy 

Production 

CR 

1158.4 

1440.6 

-282.2 

-24.3612 

97.6692 

7/25/1994 

Navy 

Production 

CR 

998.8 

900.9 

97.9 

9.801762 

79.72567 

7/25/1994 

Navy 

Production 

FP 

787 

635.1 

151.9 

19.30114 

77.19187 

7/25/1994 

Army 

R&D 

CR 

76.5 

84.6 

-8.1 

-10.5882 

94.24837 

8/25/1994 

Air  Force 

Production 

CR 

309.3 

289.9 

19.4 

6.272228 

97.51051 

8/25/1994 

Navy 

Production 

CR 

551.3 

721.6 

-170.3 

-30.8906 

117.0143 

8/25/1994 

Navy 

Production 

FP 

163.8 

142.4 

21.4 

13.06471 

95.116 

8/25/1994 

Navy 

Production 

CR 

49 

77 

-28 

-57.1429 

132.8571 

9/25/1994 

Air  Force 

Production 

CR 

134 

114.8 

19.2 

14.32836 

90.89552 

10/25/1994 

Air  Force 

Production 

CR 

897.3 

1147 

-249.7 

-27.8279 

106.7982 

10/25/1994 

Air  Force 

Production 

CR 

945.3 

984.5 

-39.2 

-4.14683 

95.10208 

10/25/1994 

Navy 

Production 

CR 

248.9 

329.1 

-80.2 

-32.2218 

117.5573 

11/25/1994 

Navy 

Production 

FP 

491 

441.7 

49.3 

10.04073 

95.21385 

12/25/1994 

Air  Force 

Production 

CR 

238.8 

239.6 

-0.8 

-0.33501 

97.52931 

12/25/1994 

Air  Force 

Production 

CR 

756.1 

764 

-7.9 

-1.04484 

96.21743 

12/25/1994 

Air  Force 

Production 

CR 

611.6 

638.5 

-26.9 

-4.3983 

89.99346 

12/25/1994 

Air  Force 

Production 

FP 

180.8 

165.5 

15.3 

8.462389 

82.68805 

12/25/1994 

Air  Force 

Production 

CR 

1447.7 

1478.5 

-30.8 

-2.12751 

94.77792 

12/25/1994 

Air  Force 

Production 

FP 

881 

993.6 

-112.6 

-12.7809 

98.29739 

12/25/1994 

Air  Force 

R&D 

CR 

215.1 

224.4 

-9.3 

-4.32357 

104.7885 

1/25/1995 

Navy 

Production 

CR 

1095.3 

1067.7 

27.6 

2.519858 

97.00539 

2/25/1995 

Navy 

Production 

FP 

853.8 

816 

37.8 

4.427266 

90.68869 

2/25/1995 

Air  Force 

R&D 

CR 

86.1 

75.7 

10.4 

12.07898 

90.82462 

3/25/1995 

Navy 

Production 

CR 

287.8 

500.4 

-212.6 

-73.8707 

155.9764 

3/25/1995 

Navy 

Production 

CR 

117.3 

93.1 

24.2 

20.63086 

78.34612 

3/25/1995 

Army 

R&D 

CR 

44.6 

50.9 

-6.3 

-14.1256 

110.3139 

4/25/1995 

Air  Force 

Production 

CR 

165.7 

167.1 

-1.4 

-0.8449 

97.40495 

4/25/1995 

Air  Force 

Production 

CR 

19.5 

16.4 

3.1 

15.89744 

100 

4/25/1995 

Navy 

Production 

FP 

228 

293.1 

-65.1 

-28.5526 

118.7719 

5/25/1995 

Army 

R&D 

FP 

285.2 

282.5 

2.7 

0.946704 

98.10659 

6/25/1995 

Army 

R&D 

CR 

72.8 

98.9 

-26.1 

-35.8516 

98.48901 

6/25/1995 

Navy 

R&D 

FP 

485.5 

672.7 

-187.2 

-38.5582 

98.4758 

7/25/1995 

Air  Force 

Production 

CR 

102.7 

102.2 

0.5 

0.486855 

99.22103 

7/25/1995 

Air  Force 

Production 

CR 

1406.1 

1400.9 

5.2 

0.369817 

97.47529 

8/25/1995 

Navy 

Production 

CR 

65.8 

122.8 

-57 

-86.6261 

168.693 

8/25/1995 

Navy 

Production 

CR 

66.6 

70.9 

-4.3 

-6.45646 

89.33934 

8/25/1995 

Army 

R&D 

CR 

93.8 

93.7 

0.1 

0.10661 

97.8678 
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SUBMITDATE 

SERVICE 

PPHASE 

CTYPE 

BAC 

ACWP 

FO 

FO% 

%COMP 

8/25/1995 

Navy 

R&D 

FP 

127.7 

183.8 

-56.1 

-43.9311 

130.0705 

9/25/1995 

Army 

Production 

CR 

373.1 

345.4 

27.7 

7.424283 

90.37791 

9/25/1995 

Navy 

Production 

CR 

264.6 

250.6 

14 

5.291005 

93.68859 

9/25/1995 

Army 

R&D 

CR 

34.3 

49.9 

-15.6 

-45.481 

140.2332 

9/25/1995 

Army 

R&D 

CR 

29 

28.7 

0.3 

1.034483 

99.31034 

10/25/1995 

Army 

Production 

CR 

236.4 

312.3 

-75.9 

-32.1066 

119.797 

10/25/1995 

Air  Force 

R&D 

FP 

89.3 

77.7 

11.6 

12.98992 

95.85666 

10/25/1995 

Navy 

R&D 

CR 

24.2 

22.7 

1.5 

6.198347 

90.90909 

10/25/1995 

Navy 

R&D 

FP 

386.3 

490.9 

-104.6 

-27.0774 

97.54077 

11/25/1995 

Air  Force 

Production 

CR 

116.1 

110.6 

5.5 

4.737295 

99.05254 

11/25/1995 

Navy 

Production 

526.8 

475.2 

51.6 

9.794989 

97.03872 

12/25/1995 

Air  Force 

R&D 

FP 

157.8 

228.4 

-70.6 

-44.7402 

98.73257 

1/25/1996 

Air  Force 

R&D 

FP 

5547.9 

8154.7 

-2606.8 

-46.9871 

93.41012 

2/25/1996 

Air  Force 

R&D 

CR 

89 

110 

-21 

-23.5955 

94.26966 

3/25/1996 

Navy 

Production 

FP 

57.6 

65.5 

-7.9 

-13.7153 

100.6944 

4/25/1996 

Air  Force 

Production 

CR 

632 

640.1 

-8.1 

-1.28165 

93.03797 

4/25/1996 

Navy 

Production 

FP 

659.5 

628.5 

31 

4.700531 

97.04321 

4/25/1996 

Air  Force 

R&D 

FP 

95.8 

125.5 

-29.7 

-31.0021 

96.55532 

4/25/1996 

Navy 

R&D 

FP 

118.9 

123.9 

-5 

-4.20521 

94.0286 

5/25/1996 

Navy 

Production 

CR 

1099.8 

1089.4 

10.4 

0.945626 

106.6285 

5/25/1996 

Navy 

Production 

CR 

965.9 

1009.6 

-43.7 

-4.52428 

100.1242 

5/25/1996 

Army 

R&D 

FP 

424.1 

418.5 

5.6 

1 .320443 

98.5145 

6/25/1996 

Army 

Production 

FP 

58.3 

56.8 

1.5 

2.572899 

98.11321 

6/25/1996 

Army 

Production 

FP 

43.8 

40.4 

3.4 

7.762557 

94.97717 

6/25/1996 

Navy 

Production 

FP 

563 

531.1 

31.9 

5.666075 

95.13321 

6/25/1996 

Navy 

R&D 

FP 

953 

1118.6 

-165.6 

-17.3767 

113.6516 

7/25/1996 

Air  Force 

Production 

FP 

1421.2 

1382.7 

38.5 

2.708978 

97.55137 

7/25/1996 

Air  Force 

R&D 

CR 

10514.3 

9751.9 

762.4 

7.251077 

88.23507 

7/25/1996 

Navy 

R&D 

FP 

17.1 

17.7 

-0.6 

-3.50877 

95.90643 

8/25/1996 

Army 

R&D 

CR 

199.6 

468.7 

-269.1 

-134.82 

224.7495 

8/25/1996 

Air  Force 

R&D 

CR 

735.3 

722.8 

12.5 

1.699986 

96.40963 

9/25/1996 

Navy 

R&D 

CR 

68.5 

60.9 

7.6 

11.09489 

88.90511 

10/25/1996 

Air  Force 

R&D 

CR 

76.2 

73.7 

2.5 

3.28084 

97.50656 

10/25/1996 

Air  Force 

R&D 

CR 

77.2 

77.1 

0.1 

0.129534 

99.6114 

10/25/1996 

Navy 

R&D 

CR 

97.6 

97.7 

-0.1 

-0.10246 

94.97951 

11/25/1996 

Air  Force 

Production 

CR 

92 

102.7 

-10.7 

-11.6304 

100.2174 

11/25/1996 

Navy 

Production 

CR 

1911 

1936.4 

-25.4 

-1.32915 

104.652 

12/25/1996 

Air  Force 

Production 

CR 

11779.5 

10895.7 

883.8 

7.502865 

93.06592 

12/25/1996 

Air  Force 

R&D 

CR 

10014.8 

19765.1 

-9750.3 

-97.3589 

196.7997 

12/25/1996 

Navy 

R&D 

CR 

21.9 

25.4 

-3.5 

-15.9817 

100 

12/25/1996 

Navy 

R&D 

CR 

24.1 

27.6 

-3.5 

-14.5228 

100.4149 

1/25/1997 

Navy 

R&D 

CR 

127.8 

132.3 

-4.5 

-3.52113 

94.2097 

2/25/1997 

Air  Force 

Production 

CR 

80.2 

157.9 

-77.7 

-96.8828 

110.0998 

2/25/1997 

Navy 

Production 

CR 

104.6 

119.6 

-15 

-14.3403 

106.5966 

2/25/1997 

Army 

R&D 

CR 

149.6 

172.6 

-23 

-15.3743 

97.59358 

4/25/1997 

Navy 

Production 

CR 

520.6 

488.9 

31.7 

6.089128 

94.98655 

5/25/1997 

Army 

Production 

CR 

189.3 

199.6 

-10.3 

-5.4411 

97.14739 

5/25/1997 

Navy 

Production 

CR 

512.7 

444.4 

68.3 

13.32163 

92.90033 
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SUBMITDATE 

SERVICE 

PPHASE 

CTYPE 

BAC 

ACWP 

FO 

FO% 

%COMP 

5/25/1997 

Navy 

Production 

CR 

706.2 

767.8 

-61.6 

-8.72274 

108.5811 

6/25/1997 

Army 

R&D 

CR 

122.5 

123.6 

-1.1 

-0.89796 

90.44898 

7/25/1997 

Navy 

Production 

FP 

561 

648.6 

-87.6 

-15.615 

100 

7/25/1997 

Navy 

Production 

FP 

329.1 

327.1 

2 

0.607718 

91.46156 

8/25/1997 

Air  Force 

Production 

CR 

94.8 

89.2 

5.6 

5.907173 

98.10127 

8/25/1997 

Air  Force 

Production 

FP 

70 

66.3 

3.7 

5.285714 

99.14286 

8/25/1997 

Air  Force 

Production 

FP 

39.3 

39.9 

-0.6 

-1.52672 

99.49109 

8/25/1997 

Air  Force 

R&D 

FP 

109.8 

85.6 

24.2 

22.04007 

95.81056 

9/25/1997 

Navy 

Production 

FP 

1060.4 

1346.6 

-286.2 

-26.9898 

105.7243 

10/25/1997 

Air  Force 

Production 

FP 

500.6 

424 

76.6 

15.30164 

96.1646 

10/25/1997 

Air  Force 

Production 

FP 

228.3 

180.8 

47.5 

20.80596 

77.48576 

11/25/1997 

Army 

Production 

FP 

26.3 

35.8 

-9.5 

-36.1217 

86.69202 

11/25/1997 

Army 

Production 

FP 

157 

149.7 

7.3 

4.649682 

93.63057 

11/25/1997 

Air  Force 

R&D 

FP 

255.6 

302.5 

-46.9 

-18.349 

98.90454 

11/25/1997 

Air  Force 

R&D 

FP 

85.9 

83.1 

2.8 

3.259604 

95.57625 

12/25/1997 

Navy 

Production 

FP 

676.5 

786.4 

-109.9 

-16.2454 

114.8263 

1/25/1998 

Navy 

Production 

FP 

632.9 

818.2 

-185.3 

-29.2779 

100.632 

3/25/1998 

Army 

R&D 

FP 

120.2 

137.2 

-17 

-14.1431 

97.33777 

3/25/1998 

Navy 

R&D 

FP 

60.9 

78 

-17.1 

-28.0788 

98.02956 

4/25/1998 

Army 

Production 

FP 

219.2 

177.5 

41.7 

19.02372 

77.82847 

4/25/1998 

Navy 

Production 

FP 

382.2 

323.5 

58.7 

15.35845 

89.76975 

5/25/1998 

Army 

R&D 

FP 

46.5 

48.8 

-2.3 

-4.94624 

99.78495 

5/25/1998 

Navy 

R&D 

FP 

233.9 

208.2 

25.7 

10.9876 

84.95083 

7/25/1998 

Army 

R&D 

FP 

55.5 

63.1 

-7.6 

-13.6937 

98.91892 

7/25/1998 

Army 

R&D 

FP 

442.8 

446.5 

-3.7 

-0.83559 

95.16712 

8/25/1998 

Army 

Production 

CR 

10.9 

9.2 

1.7 

15.59633 

86.23853 

8/25/1998 

Navy 

Production 

FP 

552.4 

534.9 

17.5 

3.167994 

97.4294 

9/25/1998 

Navy 

Production 

FP 

843.9 

919 

-75.1 

-8.89916 

95.94739 

9/25/1998 

Army 

R&D 

FP 

113.7 

99.2 

14.5 

12.75286 

86.45558 

10/25/1998 

Navy 

Production 

FP 

3188.5 

3267.6 

-79.1 

-2.48079 

98.96817 

11/25/1998 

Army 

Production 

CR 

243 

215.8 

27.2 

11.19342 

91.15226 

11/25/1998 

Air  Force 

Production 

CR 

246.8 

264.8 

-18 

-7.29335 

98.2577 

11/25/1998 

Air  Force 

Production 

CR 

129.4 

131.9 

-2.5 

-1.93199 

101.4683 

11/25/1998 

Air  Force 

Production 

CR 

421.9 

498.1 

-76.2 

-18.0612 

117.0894 

11/25/1998 

Navy 

Production 

CR 

131.5 

136.3 

-4.8 

-3.65019 

94.60076 

11/25/1998 

Navy 

Production 

FP 

283.7 

250.5 

33.2 

11.7025 

90.51815 

2/25/1999 

Navy 

Production 

CR 

683.1 

781.9 

-98.8 

-14.4635 

116.1323 

2/25/1999 

Navy 

R&D 

CR 

386.9 

411.6 

-24.7 

-6.38408 

98.13905 

3/25/1999 

Army 

R&D 

CR 

75.8 

77.2 

-1.4 

-1.84697 

94.19525 

5/25/1999 

Army 

R&D 

FP 

536 

598.2 

-62.2 

-11.6045 

99.16045 

5/25/1999 

Navy 

R&D 

FP 

300.8 

335.7 

-34.9 

-11.6024 

105.0532 

6/25/1999 

Army 

R&D 

CR 

249.5 

276.3 

-26.8 

-10.7415 

110.7415 

7/25/1999 

Air  Force 

R&D 

CR 

388.3 

388.2 

0.1 

0.025753 

98.50631 

8/25/1999 

Air  Force 

R&D 

CR 

106.7 

102.7 

4 

3.748828 

96.15745 

11/25/1999 

Army 

R&D 

FP 

118.4 

114.3 

4.1 

3.462838 

96.28378 

12/25/1999 

Navy 

R&D 

CR 

244.7 

267.7 

-23 

-9.39926 

99.95913 

2/25/2000 

Air  Force 

R&D 

CR 

130.6 

110.3 

20.3 

15.54364 

84.76263 

3/25/2000 

Air  Force 

R&D 

CR 

289.9 

265.8 

24.1 

8.313211 

89.92756 
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CTYPE 

BAC 

ACWP 

FO 

FO% 

%COMP 

4/25/2000 

Navy 

Production 

CR 

214.3 

199.8 

14.5 

6.766216 

96.82688 

5/25/2000 

Navy 

R&D 

FP 

26.7 

27 

-0.3 

-1.1236 

98.12734 

5/25/2000 

Navy 

R&D 

CR 

407.5 

438.7 

-31.2 

-7.65644 

91.0184 

6/25/2000 

Army 

R&D 

CR 

1420 

2114 

-694 

-48.8732 

144.5141 

6/25/2000 

Navy 

R&D 

CR 

25.8 

26.4 

-0.6 

-2.32558 

93.02326 

7/25/2000 

Army 

Production 

CR 

3413.3 

2618.2 

795.1 

23.29417 

76.76149 

7/25/2000 

Army 

R&D 

CR 

254.5 

280 

-25.5 

-10.0196 

98.54617 

7/25/2000 

Air  Force 

R&D 

CR 

77.1 

101.7 

-24.6 

-31.9066 

95.33074 

8/25/2000 

Air  Force 

Production 

CR 

668.9 

730.9 

-62 

-9.26895 

112.528 

8/25/2000 

Navy 

Production 

CR 

72.1 

68.3 

3.8 

5.270458 

99.58391 

8/25/2000 

Air  Force 

R&D 

CR 

76.9 

98.7 

-21.8 

-28.3485 

120.2861 

9/25/2000 

Navy 

Production 

CR 

689.5 

618.8 

70.7 

10.25381 

95.27194 

9/25/2000 

Air  Force 

Production 

CR 

58.3 

56.1 

2.2 

3.773585 

95.02573 

9/25/2000 

Navy 

R&D 

FP 

25.8 

28.1 

-2.3 

-8.91473 

98.83721 

11/25/2000 

Navy 

Production 

FP 

879.5 

1019.4 

-139.9 

-15.9068 

97.55543 

11/25/2000 

Navy 

Production 

FP 

314.6 

285.9 

28.7 

9.122695 

94.56453 

11/25/2000 

Navy 

R&D 

CR 

40.8 

40.8 

0 

0 

100 

1/25/2001 

Navy 

Production 

CR 

1567.4 

1518.1 

49.3 

3.145336 

97.37782 

1/25/2001 

Navy 

Production 

CR 

225.3 

223.3 

2 

0.887705 

98.57967 

1/25/2001 

Navy 

R&D 

CR 

3420.8 

3441.1 

-20.3 

-0.59343 

99.58197 

1/25/2001 

Navy 

R&D 

CR 

674.8 

738.1 

-63.3 

-9.38056 

100.0741 

2/25/2001 

Navy 

Production 

FP 

924.1 

975.9 

-51.8 

-5.60545 

92.20864 

2/25/2001 

Navy 

Production 

CR 

44.7 

50.6 

-5.9 

-13.1991 

90.82774 

2/25/2001 

Air  Force 

Production 

CR 

464.1 

488.3 

-24.2 

-5.21439 

106.2487 

4/25/2001 

Navy 

Production 

FP 

160.5 

213 

-52.5 

-32.7103 

123.8629 

5/25/2001 

Air  Force 

R&D 

FP 

85.3 

71.2 

14.1 

16.52989 

83.47011 

5/25/2001 

Navy 

R&D 

FP 

166.6 

163.5 

3.1 

1.860744 

94.41777 

7/25/2001 

Navy 

Production 

FP 

17.3 

16.6 

0.7 

4.046243 

78.03468 

7/25/2001 

Navy 

Production 

FP 

616.7 

597.1 

19.6 

3.178207 

95.83266 

8/25/2001 

Army 

R&D 

FP 

169.5 

176.5 

-7 

-4.12979 

97.40413 

8/25/2001 

Army 

R&D 

FP 

73.9 

82.5 

-8.6 

-11.6373 

97.29364 

9/25/2001 

Air  Force 

R&D 

FP 

3524.2 

3287.4 

236.8 

6.719255 

95.30106 

10/25/2001 

Air  Force 

Production 

FP 

395.7 

392.2 

3.5 

0.884508 

95.42583 

10/25/2001 

Navy 

Production 

CR 

42.3 

39.5 

2.8 

6.619385 

94.56265 

10/25/2001 

Navy 

Production 

CR 

481.7 

454.3 

27.4 

5.688188 

96.49159 

10/25/2001 

Navy 

Production 

CR 

520.6 

474.8 

45.8 

8.797541 

92.1821 

10/25/2001 

Navy 

Production 

CR 

460.9 

428.5 

32.4 

7.029724 

88.17531 

10/25/2001 

Navy 

Production 

CR 

1321.9 

1350 

-28.1 

-2.12573 

97.99531 

10/25/2001 

Navy 

Production 

CR 

1660.6 

1715.3 

-54.7 

-3.29399 

96.24834 

10/25/2001 

Navy 

Production 

CR 

1382.8 

1225.5 

157.3 

11.37547 

79.98988 

10/25/2001 

Navy 

Production 

FP 

2331.9 

2037.4 

294.5 

12.62919 

80.63811 

10/25/2001 

Air  Force 

Production 

FP 

75.6 

77.7 

-2.1 

-2.77778 

96.42857 

10/25/2001 

Air  Force 

Production 

FP 

263.3 

257 

6.3 

2.392708 

93.96126 

10/25/2001 

Navy 

Production 

FP 

1602.6 

1465.1 

137.5 

8.579808 

95.51354 

10/25/2001 

Army 

R&D 

FP 

616.7 

488.1 

128.6 

20.85293 

79.63353 

10/25/2001 

Army 

R&D 

FP 

171.8 

169.9 

1.9 

1.105937 

98.83586 

10/25/2001 

Army 

R&D 

FP 

69.4 

71.1 

-1.7 

-2.44957 

86.02305 

10/25/2001 

Air  Force 

R&D 

CR 

82.9 

84.8 

-1.9 

-2.29192 

100.7238 
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FO 
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10/25/2001 

Air  Force 

R&D 

CR 

539 

444.7 

94.3 

17.49536 

81.05751 

10/25/2001 

Air  Force 

R&D 

FP 

101.8 

100 

1.8 

1.768173 

98.91945 

10/25/2001 

Navy 

R&D 

FP 

3150.3 

3121.3 

29 

0.920547 

97.52405 

10/25/2001 

Navy 

R&D 

FP 

525.3 

512.8 

12.5 

2.379593 

96.24976 

11/25/2001 

Army 

Production 

CR 

124.4 

109.5 

14.9 

11.97749 

80.54662 

11/25/2001 

Air  Force 

Production 

CR 

220.7 

198.5 

22.2 

10.0589 

92.75034 

11/25/2001 

Air  Force 

Production 

CR 

351.9 

265.6 

86.3 

24.52401 

83.74538 

11/25/2001 

Navy 

Production 

FP 

919.8 

854.2 

65.6 

7.131985 

86.93194 

11/25/2001 

Navy 

Production 

FP 

1011.8 

988.7 

23.1 

2.28306 

89.97826 

11/25/2001 

Navy 

Production 

FP 

139.7 

139.6 

0.1 

0.071582 

97.56621 

11/25/2001 

Army 

R&D 

FP 

672.8 

922.9 

-250.1 

-37.173 

129.3847 

11/25/2001 

Army 

R&D 

FP 

166.9 

142.7 

24.2 

14.4997 

86.51887 

11/25/2001 

Army 

R&D 

FP 

41 

31.7 

9.3 

22.68293 

77.80488 

11/25/2001 

Army 

R&D 

FP 

135.4 

110.9 

24.5 

18.09453 

81.0192 

11/25/2001 

Air  Force 

R&D 

CR 

182 

170.4 

11.6 

6.373626 

93.46154 

11/25/2001 

Air  Force 

R&D 

FP 

331.5 

280.9 

50.6 

15.26395 

80.84465 

11/25/2001 

Navy 

R&D 

FP 

148.7 

146.7 

2 

1.34499 

99.05851 

11/25/2001 

Navy 

R&D 

FP 

196.9 

185.2 

11.7 

5.942103 

91.77247 

11/25/2001 

Navy 

R&D 

CR 

106.7 

88.5 

18.2 

17.05717 

84.44236 

11/25/2001 

Navy 

R&D 

CR 

307.1 

322.1 

-15 

-4.8844 

92.21752 

11/25/2001 

Navy 

R&D 

CR 

514.3 

527.2 

-12.9 

-2.50826 

85.08653 

11/25/2001 

Navy 

R&D 

CR 

16.1 

14.4 

1.7 

10.55901 

87.57764 

11/25/2001 

Navy 

R&D 

CR 

66.2 

59 

7.2 

10.87613 

85.04532 

11/25/2001 

Navy 

R&D 

CR 

126.1 

104.6 

21.5 

17.04996 

81.99841 

11/25/2001 

Navy 

R&D 

CR 

138.8 

133.7 

5.1 

3.674352 

85.44669 

12/25/2001 

Navy 

Production 

CR 

1161.7 

860.1 

301.6 

25.96195 

75.30343 

12/25/2001 

Army 

R&D 

CR 

78.8 

68.6 

10.2 

12.94416 

90.22843 

12/25/2001 

Army 

R&D 

CR 

210.6 

188.7 

21.9 

10.39886 

92.40266 

12/25/2001 

Army 

R&D 

CR 

82.7 

84 

-1.3 

-1.57195 

96.85611 

12/25/2001 

Air  Force 

R&D 

FP 

13447 

12842.8 

604.2 

4.493196 

92.06143 

12/25/2001 

Air  Force 

R&D 

FP 

2198.6 

2195.1 

3.5 

0.159192 

98.20795 

12/25/2001 

Air  Force 

R&D 

CR 

396.1 

349.4 

46.7 

11.78995 

87.83136 

12/25/2001 

Navy 

R&D 

FP 

285 

276.7 

8.3 

2.912281 

94.98246 
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